Introducing the Sourcemint JavaScript Loader
Christoph Dorn
production use) CommonJS package mappings based JavaScript module loader
for the browser in only 1150 bytes (minified and zipped).
See: https://github.com/sourcemint/loader-js
Demo: http://sourcemint.github.com/loader-js/workspace/www/index.html
Supported features:
* Load bundled JavaScript programs from static URLs
* Asynchronously load more program code bundles as needed
* Isolated module scopes
* Isolated package namespaces
* Isolated sandbox namespaces
* Nested and circular dependency trees
* Consistent mapping of static application resource URLs to loader
namespaces
* CommonJS/Modues/1.1
* function(require, exports, module) {}
* var ModuleAPI = require("./Module")
* CommonJS/Packages/Mappings/C (proposal)
* package.json ~ {mappings:{"PackageAlias": "PackageIdentifier"}}
* var ModuleAPI = require("PackageAlias/Module")
* CommonJS/Modues/2.0draft8 (draft)
* global.require.memoize("CanonicalModuleIdentifier", ModuleInitializer)
(no dependency argument)
* require.uri(ModuleIdentifierString) (returns ["SandboxURI",
"CanonicalModuleIdentifier"])
* (Un)CommonJS(kriskowal)/Modules
require.async(ModuleIdentifierString, function
loaded(ModuleAPI) {}, function error(e) {})
* Proposed:
* [global.]require.sandbox(SandboxURI, function loaded(sandbox) {},
SandboxOptions)
* sandbox.main()
* require.bundle("BundleIdentifier", function
ConsistentModuleSet(require) {})
Examples: https://github.com/sourcemint/loader-js/tree/master/examples
This loader combines essential CommonJS concepts into a minimal
foundation on top of which all other CommonJS concepts may be built.
I will be proposing a simpler alternative to CommonJS/Modues/2.0draft8
(draft) [1] with two primary differences:
1) Let applications ship loader plugins as part of the application
code and construct a sandbox with augmented loader to load application
into. This removes all plugin logic from the loader and ensures all
needed plugins are loaded and supported by all CommonJS loaders.
2) Do not specify dependency lists to require.memoize() as they are
not needed in the context of a bundle that includes a consistent set of
statically linked modules. Also, do not use canonical IDs when calling
require.memoize() but rather IDs that will be treated local to the
URL/PATH the module bundle is loaded from. With these changes there is
no absolute identifiers used anywhere within CommonJS module/bundle
source files allowing for much flexibility when refactoring and publishing.
Please review and let me know what you think!
Christoph
[1] - http://www.page.ca/~wes/CommonJS/modules-2.0-draft8/
Domenic Denicola
1) I'm not sure I understand the whole sandboxing concept, or its interaction with loader plugins (which are different from provider plugins?). Is there a document describing that?
2) I look forward to an updated Modules/2.0 draft spec and would love to collaborate with you on that. (Perhaps as a GitHub repository instead of a wiki page?)
3) Would it be helpful for me to adapt the NobleJS Modules/2.0 unit tests to Sourcemint, in the same fashion I did for BravoJS?
4) I am not sure why you think dependency lists are not necessary for require.memoize... Are you counting on static code analysis via Function.prototype.toString()? I feel like this ties into earlier discussions about require.memoize'd modules that depend on dynamically-provided modules (or plugins).
5) I think changing the semantics of require.uri is not necessarily wise, but introducing another method for that purpose makes sense. Maybe require.sandboxUri + require.id would combine to give the functionality you're looking for?
6) Just a style thing but... Any reason why you chose to expose sourcemint as a global, instead of as a module? Cf. require("nobleModules")
All in all, very cool, and looking forward to seeing where this goes!
Christoph Dorn
> 1) I'm not sure I understand the whole sandboxing concept, or its
> interaction with loader plugins (which are different from provider
> plugins?). Is there a document describing that?
I am changing things up a bit after having gained more practical experience.
There are no AMD-style Loader Plugins [1] because code that uses loader
plugins that are triggered by modifying the string literal passed to
require() cannot be uniformly and easily optimized when generating
bundles. Loader plugins require that:
* They are present and can be executed when generating bundles.
* Module/resource source code is bundled in a specific format
potentially leading to duplicate source code in bundles.
Also, it is not necessary to have these kinds of loader plugins at the
core loader level.
As an alternative it is trivial to load some extra convenience features
within the application to do what you need.
The whole provider plugin concept has been removed. I do not think the
way provider plugins are specified in the Modules/2 draft spec works in
real life. You cannot have two provider plugins targeting the same
endpoint and the plugin interface is too broad to make it portable.
As an alternative I propose that applications that need specific loader
"extensions" (so we don't get confused) ship these with their
application code as CJS modules and initialize a sandbox with these that
boots the actual app code. A sandbox may be modified/extended using the
third argument:
require.sandbox(url, loadedCallback, options);
A sandbox is an isolated namespace into which modules may be memoized.
There may be more than one sandbox in the same runtime. Each sandbox has
its own scoped require (which is further scoped per module) that
resolves only to modules memoized within the sandbox. Modules may reach
across sandboxes only if sandbox owner passes in reference. Sandboxes
(and all modules within them) may be destroyed.
A sandbox represents a consistent set of modules (belonging to one or
more packages) with all dependencies resolved. Packages are a thin
abstraction on the ID used to index modules within the sandbox where the
first term before the first "/" is used as a packageIdentifier.
PackageIdentifiers must be unique within each sandbox.
When applying the sandbox concept to a browser loader it allows you to
load more than one JS "application" into the same page. Each URL to a
static JavaScript boot bundle of an application is loaded into its own
sandbox allowing each application to load additional bundles into its
own sandbox or creating further nested sandboxes.
> 2) I look forward to an updated Modules/2.0 draft spec and would love
> to collaborate with you on that. (Perhaps as a GitHub repository
> instead of a wiki page?)
I will soon be maintaining my own fork to
https://github.com/kriskowal/uncommonjs
That is the best method to collaborate on specs going forward I think
(we can keep tests along with the specs).
> 3) Would it be helpful for me to adapt the NobleJS Modules/2.0 unit
> tests to Sourcemint, in the same fashion I did for BravoJS?
Absolutely. Having all tests that apply run against the loader would be
great. We can discuss how to do that here [2].
> 4) I am not sure why you think dependency lists are not necessary for
> require.memoize... Are you counting on static code analysis via
> Function.prototype.toString()? I feel like this ties into earlier
> discussions about require.memoize'd modules that depend on
> dynamically-provided modules (or plugins).
I am counting on the fact that all static links must be satisfied for
all modules within a bundle. Dynamic links (that cross bundles) can be
made with 'require.async()' and 'require.sandbox()'. Think of a bundle
as a frozen virtual filesystem. The idea is that a set of statically
linked modules can always be combined into one file which is placed into
the file that first requires the dependencies and represents the entry
point into the bundle.
It is not specified how bundles are generated. This may happen in
various ways including a build tool that uses static analysis. The PINF
[3] loader will soon generate these bundles automatically by following
static links and spidering dynamic links based on config info.
> 5) I think changing the semantics of require.uri is not necessarily
> wise, but introducing another method for that purpose makes sense.
> Maybe require.sandboxUri + require.id would combine to give the
> functionality you're looking for?
Good point/idea. I have changed it to: require.sandbox.id + require.id()
> 6) Just a style thing but... Any reason why you chose to expose
> sourcemint as a global, instead of as a module? Cf.
> require("nobleModules")
When the loader is included on a page it exports the **same** interface
to the "sourcemint" and "require" globals. If the "require" global is
already set (e.g. if requirejs is already running on the page) it will
not touch it and only export to the "sourcemint" global. This way you
can run the loader in parallel to others. The name "sourcemint" comes
from the service that will be using this loader to deliver static JS
apps via a CDN.
> All in all, very cool, and looking forward to seeing where this goes!
Thanks for the feedback! Let me know if you have any other questions. I
really think this is the way forward.
Christoph
[1] - https://github.com/amdjs/amdjs-api/wiki/Loader-Plugins
[2] - http://groups.google.com/group/pinf-dev/
[3] - https://github.com/pinf/loader-js