Announcing Noble Modules: a Modules/2.0 implementation

[Domenic]

As I've mentioned in a few threads, my team has been working on a
Modules/2.0 implementation. Well, now it's ready!

https://github.com/NobleJS/Noble-Modules

Noble Modules is used, in a nascent form, in bn.com's NOOK Study [1]
and NOOK for Mac [2] applications. Upcoming releases of both these and
NOOK for PC will be using the fully Modules/2.0-compliant version
released today. So in this sense, Modules/2.0 has quite a large user
base already :D.

We believe that Modules/2.0 is the way forward for browser-based
CommonJS-conformant JavaScript modules, and hope we've contributed to
the community and to Modules/2.0's forward momentum with this release.
We'd love your feedback, and appreciate the help you've already given
on this mailing list with hammering out the all-important corner
cases.

Cheers,
-Domenic

[1]: http://www.barnesandnoble.com/nookstudy/index.asp
[2]: http://www.barnesandnoble.com/u/nook-for-mac/379002740/

[Wes Garland]

Hi, Domenic!

Congratulations on the implementation and release of Noble Modules!

Sorry I haven't been active in here as of late: I am *completely* swamped with a new project, but still very much doing CommonJS on a daily basis  (new project is written in GPSEE + BravoJS: we may very well evaluate Noble Modules in place of BravoJS mid-phase).

re. your last message -- any information you can provide on BravoJS failures during TDD would be very useful feedback.  Will also be reviewing your last communication and Noble Modules before I take a stab at Modules/2.0 draft 9: and I expect draft 9, at this stage, to simply be a clarification of unresolved issues or errors in draft 8.

Wes

--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

[Christoph Dorn]

Looks great! Thank you for this. I'll provide more feedback as I work 
with it.

I look forward to adding CommonJS/Package support and compiling PINF 
programs to run on NobleJS in the browser.

Once that is done I think I am ready to propose a package, mappings & 
transport standard for Modules 2 that spans the server and browser 
environments.

Christoph

Domenic 22 August, 2011 12:48 PM

[Domenic]

Thanks so much, both of you! It's really heartening to get such
supportive responses :D

Wes, reorganizing the tests and the test runner a bit so that I can
easily apply them to other Modules/2.0 implementations is high on my
priority list. Hopefully by the end of the weekend I'll be able to run
them against BravoJS and let you know exactly where the discrepancies
are. It's great to hear that you're planning on a draft 9, too!

Christoph, great to hear that this helps your efforts! It'll be
interesting to see how PINF can be built on top of Noble Modules,
since from what I recall of the source code it has a number of hooks
into BravoJS internals (e.g. in your custom plugin layer). I'd imagine
doing so could reveal some weaknesses in the spec, i.e. things that
you can currently do with BravoJS but cannot do with a general Modules/
2.0 implementation. It'd be cool to find those and see if we want to
add them to the spec, or come up with alternative ways of achieving
the same thing within the existing capabilities.

-Domenic

[Christoph Dorn]

Domenic 25 August, 2011 12:00 AM

Christoph, great to hear that this helps your efforts! It'll be
interesting to see how PINF can be built on top of Noble Modules,
since from what I recall of the source code it has a number of hooks
into BravoJS internals (e.g. in your custom plugin layer). I'd imagine
doing so could reveal some weaknesses in the spec, i.e. things that
you can currently do with BravoJS but cannot do with a general Modules/
2.0 implementation. It'd be cool to find those and see if we want to
add them to the spec, or come up with alternative ways of achieving
the same thing within the existing capabilities.

The PINF <-> BravoJS interface is as clean as I could get it while working on the quite complex problem of cross-platform/engine and environment portability and staying as spec compliant as possible.

Having an alternative to BravoJS will now allow me to take a deeper look at the various points of integration and boundaries.

I see it happening in the following order:

  1) Add CommonJS package and mappings support to Noble Modules (as a plugin)
  2) Revise module transport format
  3) Adjust the PINF Program Server [1] to serve CommonJS packages (modules within them) in the revised transport format and optionally use BravoJS or Noble Modules to run the program in the browser.
  4) Refactor the PINF JavaScript Loader to optionally use Noble Modules instead of BravoJS internally. This will be the biggest challenge and result in changes for BravoJS and Noble Modules but should give us a very clear idea for what is needed in terms of plugins at different levels.

(1), (2) and (3) are pretty straight forward.

(4) will take some real time and I am not sure when I will get to that. I hope before Modules/2 draft goes final which would be ideal but that may be unrealistic.

Once all this work is done we will have an implemented foundation for interoperability as we will have two minimal loaders (BravoJS and Noble) that can be used interchangeably with a plugin system that is sufficient and portable. I see Modules/2 gaining real momentum once this is done. I should have a package registry and docs site up by then which can test packages against the spec and run automated tests which should add fuel to the fire of adoption.

Very exiting!

Christoph

[1] - https://github.com/pinf/loader-js/tree/master/demos/ProgramServer

[Domenic]

All right, I got the Noble Modules unit tests running against BravoJS:

https://github.com/NobleJS/Noble-Modules/tree/testing-bravojs

The failures generally seem to fall into a few categories, ordered
from least severe to most:

1) Minor/easy-to-fix spec conformance issues, like module.id should be
undefined in the EME, require.uri should exist, require.memoize should
throw an error when a module is already memoized.

2) Labels don't seem to work; for a dependency array of [{ mathLabel:
"demos/math" }], require("mathLabel") made BravoJS go searching for
mathLabel.js.

3) Relative paths do not seem to work: BravoJS interprets them all
relative to the main module. E.g. if demos/area has "./math" in its
dependency array, BravoJS goes looking for "<mainModuleDir>/math.js"
instead of "<mainModuleDir>/demos/math.js".

4) Issues with the robustness of module provision and its interaction
with memoization. Although it is hard to tell which of these fail
because of the use of relative paths, and which fail because of issues
with provision, it seems scenarios like a diamond dependency tree or
providing the same module twice can trip up BravoJS. And the
discussion at http://groups.google.com/group/commonjs/browse_thread/thread/53057f785c6f5ceb
about provision interaction with memoization leads to several test
cases failing.

5) Major differences in our interpretation of the responsibility of
load vs. provide. Essentially, the spec does not indicate that
module.load, or the module.declare call that it results in, should
memoize the module in question. It seems strictly to be intended for
loading the <script> tag and executing the resulting module.declare.
This causes all of the provider plug-in tests to fail, as well as the
one explicitly testing that module.load does not memoize. I think that
the spec should stay as-is in this regard, and perhaps call out the
necessity of not memoizing the module, because---as evidenced by the
plug-ins included with Noble Modules---doing so makes plug-in writing
MUCH easier, as alternate script-loading can be done simply by
overriding module.load to fulfill the specified duties, and leaving
module.declare alone. Compare:

https://github.com/NobleJS/Noble-Modules/blob/master/naked.js
http://code.google.com/p/bravojs/source/browse/plugins/jquery-loader/jquery-loader.js

--

As another thing I noticed, initially my tests were failing because
require.memoize and require.isMemoized expect canonical module IDs,
instead of module identifiers. Is there a reason for this, especially
in the case of isMemoized? Since they have access to the local
context, they could just as easily accept identifiers, allowing
require.isMemoized("x") instead of
require.isMemoized(require.id("x")). And it's weird to do
require.memoize("x", ...) but then get a failure when you immediately
do require("x"); there will end up being a lot of boilerplate
require.memoize(require.id("jQuery"), [], function () { return
jQuery; }) in the wild, which could be reduced to
require.memoize("jQuery", [], function () { return jQuery; }).

Hope this is helpful!

[Wes Garland]

> Hope this is helpful!

*extremely* helpful.  BravoJS and a significant chunk of spec both suffer from single-author syndrome.  The next time I have time to hack the spec and BravoJS  (they are hand-in-hand for me), I will absolutely be addressing each of these cases.  I recognized a few bugs just in your description, and your call-out w.r.t. module.provide vs. module.load sounds like potentially insufficient thinking on the specification on my part.

Wes

--

You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.