"exports.foo = foo" or "provide('foo', foo)" for Securable Modules
Peter Michaux
Securable Modules proposal.
exports.foo = foo
or
provide('foo', foo)
If we go with augmenting an exports object, I think it should just be
a plain JavaScript object and that it should also be known to be the
object returned by "require". This make the use of assignment syntax
intuitive and also makes the relationship between "exports" and
"require" easy to understand. I'm not in favor of "exports" becoming
an oddball host object and having a new clone returned by each call to
"require". That is too confusing. I'm not even in favor of leaving
that open as a possibility. I want a guarantee that "exports" is
returned by "require".
If regular assignment behavior is not desired, and that only an owner
module can affect its exports, then I think we should go with the
"provide" function so it is a clear abstraction and the programmer
cannot make assumptions about how it works based on the syntax. I
could be happy with this solution also.
Peter
Kris Zyp
Peter Michaux wrote:
> It seems there are two ways floating around for making exports in the
> Securable Modules proposal.
>
> exports.foo = foo
>
+1 for this style. From the server side, it's easy enough to freeze the
object (make it immutable) prior to returning it from the "require"
function call for those that want to (I would probably be one of those).
Kris
Robert Cerny
Robert
ihab...@gmail.com
On Tue, Feb 10, 2009 at 1:23 PM, Kris Zyp <kri...@gmail.com> wrote:
> +1 for this style. From the server side, it's easy enough to freeze the
> object (make it immutable) prior to returning it from the "require"
> function call for those that want to (I would probably be one of those).
That's a really cool option. However, here's a counter-scenario:
1. You wish to build a securable system; and
2. You have a circular dependency.
A module probably does something like:
// In module "A"
exports.a = ...;
...
require("B");
...
exports.b = ...;
...
Object.prototype.freeze(exports);
If module B depends back on A, it will gain access to A's exports
before A has a chance to freeze them. This is a potentially subtle
pitfall.
Now, there is a workaround, which is for module A to do:
// In module "A"
exports.a = ...;
exports.b = ...;
Object.prototype.freeze(exports);
...
require("B");
...
which means that, if one of the exports of A relied on stuff from B, A
must create some sort of a proxy to that stuff, assign it to exports,
freeze the lot, and *then* populate that proxy with the real stuff.
Like [*]:
var theB = undefined;
exports.b = function(x, y, z) {
theB(x, y, z);
};
require("B");
theB = /* the real stuff */;
Ihab
--
Ihab A.B. Awad, Palo Alto, CA
Kris Zyp
Yeah, I had considered that. I could also have the require function
freeze the exports while other requires are executing, and then unfreeze
when back in the original require function. For Persevere, I wasn't
going to worry about that though, I was just going to let the modules
battle it out amongst themselves and then freeze exports for the code
that is returned to the application code (which comes from a completely
different place in Persevere, it's stored in structured class definition
units associated with DB tables). In reality, one of my prime
motivations for immutability is avoiding concurrency issues, and so all
the libraries are loaded and their exports will be frozen in single
thread before the multiple threads of the HTTP request engine starts in
on it.
Kris
ihab...@gmail.com
On Tue, Feb 10, 2009 at 1:51 PM, Kris Zyp <kri...@gmail.com> wrote:
> ... I could also have the require function freeze the exports while other
> requires are executing, and then unfreeze when back in the original
> require function.
If you mean ES 3.1+ freezing, that unfortunately does not work:
freezing is 1-way.
> For Persevere, I wasn't going to worry about that though, I was just
> going to let the modules battle it out amongst themselves and then freeze
> exports for the code that is returned to the application code ...
Right, but the hard question is really when the application code
itself is modularized.
> In reality, one of my prime motivations for immutability is avoiding concurrency
> issues ...
That's a noble and reasonable goal. :)
The prime motivation for me is to support the possibility of
correctness. In other words, if modules A and B both depend on module
C, it should be possible for C to behave in such a way that it does
not create a way for A to attack B, or either A or B to attack C.
As we've both illustrated, this *is* possible using the scheme of
"create proxies, freeze, then export" but leads to very difficult and
error-prone coding. I'm just worried about that part of it.
Kris Zyp
ihab...@gmail.com wrote:
> Hi Kris,
>
> On Tue, Feb 10, 2009 at 1:51 PM, Kris Zyp <kri...@gmail.com> wrote:
>
>> ... I could also have the require function freeze the exports while other
>> requires are executing, and then unfreeze when back in the original
>> require function.
>>
>
> If you mean ES 3.1+ freezing, that unfortunately does not work:
> freezing is 1-way.
>
>
Yes, I know ES3.1+ freezing is one-way, I would do it as a host object
that controls property modifications based on the lifecycle. However,
know I am wondering if maybe ES3.1 freezing is enough (see below)...
>
>> In reality, one of my prime motivations for immutability is avoiding concurrency
>> issues ...
>>
>
> That's a noble and reasonable goal. :)
>
> The prime motivation for me is to support the possibility of
> correctness. In other words, if modules A and B both depend on module
> C, it should be possible for C to behave in such a way that it does
> not create a way for A to attack B, or either A or B to attack C.
>
If "require" freezes it's exports when a module is finished executing,
then the only way for A to mutate B's exports ("attack" B) is if B
requires A, as that is the only opportunity for A to access B's exports
prior to them being frozen. However, the attack that is being launched
is a modifying B's behavior. But if B already depends on A for it's
functionality, A can probably already do such an attack simply by not
delivering the functionality that B depends on. For example, if I depend
on File for my functionality, it seems to hard to avoid the fact that if
File doesn't deliver file access functionality, I am hosed anyway, File
doesn't need to try to modify my exports to attack me.
Anyway, maybe I missing something here, but this doesn't seem like a
significant security flaw, IIUC.
Kris
ihab...@gmail.com
> If "require" freezes it's exports when a module is finished executing,
> then the only way for A to mutate B's exports ("attack" B) is if B
> requires A, as that is the only opportunity for A to access B's exports
> prior to them being frozen. However, the attack that is being launched
> is a modifying B's behavior. But if B already depends on A for it's
> functionality, A can probably already do such an attack simply by not
> delivering the functionality that B depends on.
Agreed -- *but* B can rely on A in a controlled manner. B can use A's
services and still be suspicious of A.
Mark Miller
If "require" freezes it's exports when a module is finished executing,
then the only way for A to mutate B's exports ("attack" B) is if B
requires A, as that is the only opportunity for A to access B's exports
prior to them being frozen.
--
Text by me above is hereby placed in the public domain
Cheers,
--MarkM
Kevin Dangoor
something and that we all understand the tradeoffs we're talking
about.
On Tue, Feb 10, 2009 at 4:03 PM, Peter Michaux <peterm...@gmail.com> wrote:
>
> It seems there are two ways floating around for making exports in the
> Securable Modules proposal.
>
> exports.foo = foo
And, the other side:
bar = require("foomodule");
bar.foo();
Pros:
* easy to use and understand
* closest to how people do modules in JS today
* module objects are mutable and can be made mostly immutable by shallow copying
Cons:
* Shallow copying or some other mechanism of immutability creates JS
objects that are not what people are used to
* Circular dependencies will lead to an exception
> or
>
> provide('foo', foo)
bar = require("foomodule");
bar('foo')();
Pros:
* The module machinery lives entirely in functions, thus making the
behavior of exporting and importing completely controlled
* Modules are certain to not be able to attack each other
* Circular dependencies are not a problem
Cons:
* Less pleasant syntax bar('foo')() vs. bar.foo()
* Modules are immutable in all cases, short of having another function
that allows you to change things in a module, which seems unpleasant
* Modules are less like what people are used to
My concern here is that the case of module A attacking module B is not
that much of a concern in application scripting. (In fact, it's common
practice in Ruby ;). I would rather have secured environments need to
lock down their exports object than have every server side JS
programmer using a less pleasant module model.
It's hard to underestimate the importance of how the syntax feels...
after all, that's the "user interface" of what we're building here.
Circular dependencies in practice are not that much of a problem in practice.
Kevin
--
Kevin Dangoor
work: http://labs.mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com
Robert Cerny
> And, the other side:
>
> bar = require("foomodule");
> bar.foo();
>
> Pros:
>
> * easy to use and understand
> * closest to how people do modules in JS today
> * module objects are mutable and can be made mostly immutable by
> shallow copying
This looks good to me. Shallow copying means to me that on every call
to require there is a new shallow copy given out. This copy, as far
as i see it, behaves just like any other JavaScript object. Thus, a
module who wishes to make some adjustments of a imported module can
do so in complete privacy, best by using assignments directly to the
module and not to any product of it, so others stay unaffected.
I think this is a really good solution. Am i overlooking something?
Robert
Peter Michaux
var A = require('A');
exports.bar = function() {
return A.foo();
};
If "foo" has not been exported at the time A is required (and its
exports shallow copied) then the call "A.foo" fails. This sort of
shallow copy brakes down the dynamic nature of JavaScript.
With shallow copying, the module programmer will need to jump through
hoops to get that dynamic nature back. We will see a lot of the
following to ensure the latest exports are used (and it is not
thread-safe)
exports.bar = function() {
return require('A').foo();
};
The above verbosity would be sprinkled everywhere because it will be
"a best practice".
----
I think whatever is returned by "require" should live-update as the
exports of the module are updated.
Peter
ihab...@gmail.com
On Wed, Feb 11, 2009 at 11:24 AM, Peter Michaux <peterm...@gmail.com> wrote:
> var A = require('A');
>
> exports.bar = function() {
> return A.foo();
> };
>
> If "foo" has not been exported at the time A is required (and its
> exports shallow copied) then the call "A.foo" fails.
Right. I think the assumptions of the snapshotting scheme (which I
state without bias, but for clarification) are:
1. Module A does its darndest (yes, that is a technical term of art,
and thank you) to ensure that, by the time its module function is
complete, all its symbols are attached to its exports.
2. If Module A wishes to export a dynamically updating object, it
exports *that* as a separate symbol rather than trying to dynamically
update the exports object itself. Encouraging this usage thus
simplifies the common pattern where a module exposes some symbols
which are just all *available* at the time of require()-ing.
3. Module A may fail to do #1 in the case of cyclic dependencies, at
which point the snapshot policy allows its clients to at least get a
partial set of its imports.
4. Module A can dynamically update its "exports" and expect its
clients to re-require() over and over if it really wants to but, given
facilities #1 - #3, it would be really weird, and somewhat bootless,
if also harmless.
Kevin Dangoor
> With shallow copying, the module programmer will need to jump through
> hoops to get that dynamic nature back. We will see a lot of the
> following to ensure the latest exports are used (and it is not
> thread-safe)
>
> exports.bar = function() {
> return require('A').foo();
> };
>
> The above verbosity would be sprinkled everywhere because it will be
> "a best practice".
To be clear, I'm not advocating shallow copying for general use. I'm
just saying that if someone wants to get this sort of immutability *in
their own systems* then they can go for it. I'd rather see most folks
just working with normal, mutable objects.
Peter Michaux
>
> On Wed, Feb 11, 2009 at 2:24 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> With shallow copying, the module programmer will need to jump through
>> hoops to get that dynamic nature back. We will see a lot of the
>> following to ensure the latest exports are used (and it is not
>> thread-safe)
>>
>> exports.bar = function() {
>> return require('A').foo();
>> };
>>
>> The above verbosity would be sprinkled everywhere because it will be
>> "a best practice".
>
> To be clear, I'm not advocating shallow copying for general use. I'm
> just saying that if someone wants to get this sort of immutability *in
> their own systems* then they can go for it. I'd rather see most folks
> just working with normal, mutable objects.
The issue is what the spec guarantees.
If the spec guarantees the object returned by "require" is the
"exports" object then we can program a particular way assuming late
binding.
If that guarantee is not made we must program as though it is not the
same object (even if it is in some implementations) to avoid any
unexpected surprises. Programming this way is less than desirable, in
my opinion.
Peter
Robert Cerny
> var A = require('A');
>
> exports.bar = function() {
> return A.foo();
> };
>
> If "foo" has not been exported at the time A is required (and its
> exports shallow copied) then the call "A.foo" fails. This sort of
> shallow copy brakes down the dynamic nature of JavaScript.
I would expect a module to have all its product available for me when
i require it. Basically exports should be in the top level block of
the module, all collected at the end of the module one by one, so it
is easy to see what it offers.
Peter, can you give me an example of module that would *need* to
change it exports over time?
Robert
Peter Michaux
> Peter, can you give me an example of module that would *need* to
> change it exports over time?
No; however, I think it is in the character of a dynamic language.
Being able to pass around the object returned by "require" as a live
representation of a module's exports seems like a good idea.
There is nothing in JavaScript like the "snapshot" or "shallow copy"
idea and I don't see the need to add such an idea.
Peter
Robert Cerny
> On Wed, Feb 11, 2009 at 11:45 AM, Robert Cerny
> <robert...@gmail.com> wrote:
>
>> Peter, can you give me an example of module that would *need* to
>> change it exports over time?
>
> No; however, I think it is in the character of a dynamic language.
> Being able to pass around the object returned by "require" as a live
> representation of a module's exports seems like a good idea.
If we do not protect the module object, we run into the same name
collision issues that we have in global. The only difference is how
the object is accessed. I thought we tried to avoid stepping on each
others toes?
Robert
Peter Michaux
>
> On Feb 11, 2009, at 8:54 PM, Peter Michaux wrote:
>
>> On Wed, Feb 11, 2009 at 11:45 AM, Robert Cerny
>> <robert...@gmail.com> wrote:
>>
>>> Peter, can you give me an example of module that would *need* to
>>> change it exports over time?
>>
>> No; however, I think it is in the character of a dynamic language.
>> Being able to pass around the object returned by "require" as a live
>> representation of a module's exports seems like a good idea.
>
> If we do not protect the module object, we run into the same name
> collision issues that we have in global.
Not quite the same. With the Pythonic or Securable Modules proposal,
it would take an intentional move by a module to step on another
module. With the global eval system either non-disciplined programming
or outright mistakes (forgetting var) can cause trouble. I believe
that some people here are looking for these circumstances to be
handled well and still allow intentional stepping on another module.
> The only difference is how
> the object is accessed.
Yes and unfortunately once again in the programming world, syntax is
playing a role in the decision making. Folks love their dot notation
and allowing that plus read-only access means the foreign concept of
"snapshots" must be introduced. I think that is the tail wagging the
dog.
> I thought we tried to avoid stepping on each
> others toes?
We do but should we be allowed to if we want to?
Peter
Kris Kowal
> return require('A').foo();
> };
>
> The above verbosity would be sprinkled everywhere because it will be
> "a best practice".
This is not true. It would be sufficient, in the odd cyclic
dependency case where it's necessary, to say:
<A>
exports.foo = function () {
return B.bar();
};
var B = require('B'); // cyclic dependencies on the bottom
<B>
exports.bar = function () {
return A.foo();
};
var A = require('A'); // cyclic dependencies on the bottom
However, I presently am willing to compromise: returning the exact
"exports" object as was used to construct the module singleton since I
would prefer to see convergence rather than have pesky edge case
details derail this effort. I personally would never modify the
exports of a module I request, but that doesn't mean that someone else
might not want to be an expedient jerk. I agree with Robert that
doing so would partially defeat the purpose of using modules, and am
intrigued by his observation that, if each module got a snapshot of
the exports, they would be able to modify that snapshot to their
liking. But, it's true that the motivations for snap shooting the
module and freezing it are not strictly for security reasons, but to
"tame" or "dominate" the language, which is not strictly in agreement
with the "rhinocerous" spirit of the language. We can retreat to the
sandbox for defining security boundaries.
Kris Kowal
Robert Cerny
> Not quite the same. With the Pythonic or Securable Modules proposal,
> it would take an intentional move by a module to step on another
> module.
No, the move can also be unintentional. Two different authors may
attach an something by the same name to "augment" a module.
Furthermore, if i use a lot of modules in my application, i cannot be
sure that the module A behaves a documented because some module B
might modify its behavior in silence. The differences i see between
global object loading and the "secureable" modules are
* the manner in which objects are referred to: by means of require
instead of the name under global
* how objects are organized: in a module cache instead of below global
* how modules export products: by attaching it to an object in the
module tree instead of attaching it to an object under global
Robert
Peter Michaux
>
> On Feb 11, 2009, at 9:44 PM, Peter Michaux wrote:
>
>> Not quite the same. With the Pythonic or Securable Modules proposal,
>> it would take an intentional move by a module to step on another
>> module.
>
> No, the move can also be unintentional. Two different authors may
> attach an something by the same name to "augment" a module.
True.
> Furthermore, if i use a lot of modules in my application, i cannot be
> sure that the module A behaves a documented because some module B
> might modify its behavior in silence.
True.
As I've written, I could support the idea that a module's exports are
not mutable by other modules. That is a sound programming idea: a
module controls everything coming into it and going out of it. This
can be done with abstraction using the functions approach of "provide"
to export and "require" returning a getter function.
I don't support the snapshots idea.
Peter
ihab...@gmail.com
> As I've written, I could support the idea that a module's exports are
> not mutable by other modules. That is a sound programming idea: a
> module controls everything coming into it and going out of it. This
> can be done with abstraction using the functions approach of "provide"
> to export and "require" returning a getter function. I don't support the
> snapshots idea.
My personal standpoint is this:
0. I am agnostic re snapshots vs. functions.
1. I would favor the consistent and dead-doggone-simple approach is
what Kris Kowal agreed to in a different thread: to simply return the
exact module exports to all require()-ers, without any intermediation.
2. With the approach of #1, it is possible to write a defensively
consistent module by doing the following phases (per my discussion
on-list with Kris Zyp). Since this is the uncommon case, it is okay
for that to be a bit long-winded:
// Declare empty, lexically scoped, protected state
var theFunc = undefined;
// Assign stubs to exports, then freeze exports
exports.func = function(x, y, z) { theFunc(x, y, z); }
Object.prototype.freeze(exports);
// Require() all the required stuff
require('foo.js'); require('bar.js'); ...
// Fill in the protected state, using require()-ed stuff
theFunc = function(x, y, z) { /* do real work */ }
3. With this approach, any monkey patching design patterns remain
well-supported.
4. The unit of defensive consistency remains the sandbox, and the
style of programming for security is to create a sandbox for each
module instance, explicitly specifying the environment for each
instance.
5. In case #4, if the module thusly loaded *itself* does not use the
sandbox/singleton require(), it does not have to worry about cyclic
dependencies. It can just freeze its exports at the end of its
execution and return a frozen, defensively consistent value to its
clients.
Does this all make sense?
Peter Michaux
> 1. I would favor the consistent and dead-doggone-simple approach is
> what Kris Kowal agreed to in a different thread: to simply return the
> exact module exports to all require()-ers, without any intermediation.
>
> 2. With the approach of #1, it is possible to write a defensively
> consistent module by doing the following phases (per my discussion
> on-list with Kris Zyp). Since this is the uncommon case, it is okay
> for that to be a bit long-winded:
>
> // Declare empty, lexically scoped, protected state
> var theFunc = undefined;
>
> // Assign stubs to exports, then freeze exports
> exports.func = function(x, y, z) { theFunc(x, y, z); }
> Object.prototype.freeze(exports);
>
> // Require() all the required stuff
> require('foo.js'); require('bar.js'); ...
>
> // Fill in the protected state, using require()-ed stuff
> theFunc = function(x, y, z) { /* do real work */ }
Indeed this is possible but this doesn't give Robert Cerny the
guarantee he wants. He wants to know that if he includes a module that
it has not been monkey patched at all. Every module author won't
necessarily code in the above style and that brakes the possibility of
that guarantee. I can understand his desire in a big system of code
where some library written a long time ago does some monkey patching.
Then another separate body of code is merged into the project that
depends on monkey patching not being there (or even worse does its own
overlapping monkey patching.)
On the other hand, if we want monkey patching available, then coding
in the above style goes against that goal.
---
The problem can be approached the other way around. Modules can be
immutable and the module author can open up mutability if desired with
getFoo and setFoo functions being exported by the module. This is
better from a security standpoint as immutable by default is like
private by default.
---
I think Kevin Dangoor's question still stands
"*Should* the modules be immutable?"
With that question answered we can design a solution.
Peter
Kevin Dangoor
>
> On Feb 11, 2009, at 9:44 PM, Peter Michaux wrote:
>
>> Not quite the same. With the Pythonic or Securable Modules proposal,
>> it would take an intentional move by a module to step on another
>> module.
>
> No, the move can also be unintentional. Two different authors may
> attach an something by the same name to "augment" a module.
> Furthermore, if i use a lot of modules in my application, i cannot be
> sure that the module A behaves a documented because some module B
> might modify its behavior in silence.
In my experience, this really doesn't happen in practice. There are
thousands of packages in the Python package index and very few that
I've encountered do any sort of monkeypatching. It's rare enough in
the context of libraries that you'll get and install that it would
likely be documented. I'd guess that the experience of the Dojo people
would be similar (I certainly never had such issues when I used Dojo).
I believe that a language/API should protect against really common
errors (leaving off "var" for example, is quite easy), but should
otherwise assume that the person writing their code knows what they're
doing. In some ways, JavaScript gives more freedom than Python because
of the flexibility of prototypes and the ability to do things like
extend Object.prototype.
To me, eliminating monkeypatchability removes flexibility in order to
fix a problem that does not come up in practice.
(It's like the argument against operator overloading: "it can be
abused!" Sure, it *can*... but it can also be used to give you things
like a convenient dictionary syntax. I'd rather have that than worry
about the abuses.)
Kevin
p.s. as a side note, there have been times in the past where I've
found it more convenient to maintain a monkeypatch against a library
(until the library is changed) instead of a traditional patch.
Mark Miller
On Wed, Feb 11, 2009 at 2:47 PM, <ihab...@gmail.com> wrote:
> Object.prototype.freeze(exports);
That's "Object.freeze(exports);" The ES3.1 effort was very careful to avoid adding anything to Object.prototype.
I think Kevin Dangoor's question still stands
"*Should* the modules be immutable?"
Kris Zyp
Kevin Dangoor wrote:
> On Wed, Feb 11, 2009 at 5:18 PM, Robert Cerny <robert...@gmail.com> wrote:
>
>> On Feb 11, 2009, at 9:44 PM, Peter Michaux wrote:
>>
>>
>>> Not quite the same. With the Pythonic or Securable Modules proposal,
>>> it would take an intentional move by a module to step on another
>>> module.
>>>
>> No, the move can also be unintentional. Two different authors may
>> attach an something by the same name to "augment" a module.
>> Furthermore, if i use a lot of modules in my application, i cannot be
>> sure that the module A behaves a documented because some module B
>> might modify its behavior in silence.
>>
>
> In my experience, this really doesn't happen in practice. There are
> thousands of packages in the Python package index and very few that
> I've encountered do any sort of monkeypatching. It's rare enough in
> the context of libraries that you'll get and install that it would
> likely be documented. I'd guess that the experience of the Dojo people
> would be similar (I certainly never had such issues when I used Dojo).
>
Dojo does replace functions on objects in order to provide it's AOP
features (dojo.connect provides after advice, so you can listen for
calls), but this doesn't affect the functionality of the original
function. I am not sure if this is categorized as "monkeypatching" by
those who like to make sweeping generalizations of programming practices.
Kris
Mark Miller
Dojo does replace functions on objects in order to provide it's AOP
features (dojo.connect provides after advice, so you can listen for
calls), but this doesn't affect the functionality of the original
function. I am not sure if this is categorized as "monkeypatching" by
those who like to make sweeping generalizations of programming practices.
Kevin Dangoor
> Dojo does replace functions on objects in order to provide it's AOP
> features (dojo.connect provides after advice, so you can listen for
> calls), but this doesn't affect the functionality of the original
> function. I am not sure if this is categorized as "monkeypatching" by
> those who like to make sweeping generalizations of programming practices.
That is actually a legitimate use for this that I hadn't thought of,
whether you call it monkeypatching or not.
Kevin