draft of dependency loading scheme (any number of dep's in 2http calls)

[SamuraiJack]

Source data:

Module1 uses Module2, Module3 as dependecies
Module2 uses Module4, Module5
Module3 uses Module5
Module4 uses Module6

this info is reflected in sources

=====
1st we need to determine the full list of "in-depth dependencies" for
each module (and modify its sources with computed list)

that will be:
Module1 uses Module2, Module3, Module4, Module5, Module6
Module2 uses Module4, Module5, Module6
Module3 uses Module5
Module4 uses Module6


this should be done statically - for example during deployment stage

=====
Then, lets say we need to load Module1 dynamically.

1. There is no prior knowledge about its dependencies, so we use the
1st http call to load Module1
2. After loading Module1 we got full info about its dependencies -
[Module2, Module3, Module4, Module5, Module6] list
Lets imagine, that Module5 is already loaded.
So we exclude Module5 from that list
[Module2, Module3, Module4, Module6]
then stringifying it, and computing MD5 hash, let it be =ABC123

Then we making 2nd request, like this:
/libmanager/abc123.js
with setuped header "X-Joose-use" = [Module2, Module3, Module4,
Module6]

if the manager already have abc123.js file ready - its serves as
static file.
otherwise - libmanager builds a new package, based on "X-Joose-use"
header content.

[Malte Ubl]

Hey,

here are some strategies for loading JS:

Page A needs JS file 1,2,3,4,5
Page B needs JS file 2,3,4,5,6

In order to minimize http requests, the strategies would be to

Strategy X:
1. Visit page A , load a package containing all needed JS files
2. Visit page B, once again load a package containing all needed JS files

plus: Simple
minus: Loading too much data

Strategy Y
1. Visit page A, load a package containing all needed JS files
2. Visit page B, somehow determine that we were already in Page A and
use that package plus and extra request for file 6

plus: Minimal data
minus: Very complex, needs support on the server for dynamically
assembling JS packages.

Strategy Z:

1. Visit page A Load a package containing all needed JS files for the entire App

plus: Simple
plus: Maximum result from zip-compression
minus: User loads code which he might not need eventually

My current thinking is, that if (and only if) the package resulting
from strategy Z is not "too large" it is the best strategy most of the
time.

If it is possible to easily determine disjunct packages of JS files,
it makes of course sense to load these separately. In this case, it
is, however, highly likely that we are talking about separate
applications anyway.

Bye
Malte

[Nickolay Platonov]

imho the Y strategy is the most promising, coz both the X and Z are the edge cases.

[Jeremy Wall]

Actually I'm a big fan of doing the simplest thing first and then optimizing if needed. so X appeals the most to me.

[Malte Ubl]

Sorry, accidently only replied to Jeremy.


---------- Forwarded message ----------
From: Malte Ubl <malt...@gmail.com>
Date: Mon, Dec 1, 2008 at 9:27 PM
Subject: Re: draft of dependency loading scheme (any number of dep's
in 2http calls)
To: Jeremy Wall <jw...@google.com>


Sounds right. In my opinion strategy Z might actually be easier,
because it does not require any knowledge about the application.

My perfect solution, would
- embed the dependency knowledge within the Joose Module definition
- allow integration of non-Joose components like jQuery, etc.
- allow integration of libraries which reside at an URL (major
libraries delivered by CDNs)
- run on Windows, MacOS X and Linux

This pretty much means, we'd have to go with Rhino as a base and go from there.

I really hate that there is no CPAN for JavaScript, just as much as I
hate that there is no CPAN for Java (Maven might come close), .NET,
PHP, ruby (gems come close but dont get there). Aiming for completing
JSAN probably aims too high.

Bye
Malte

[Jeremy Wall]

Yeah start simple and go from there. The bare minimum you need is:
 - a repository of modules stored by version. with an index.
 - a way to determine dependencies for a module.
 - something that can retrieve the appropriate module and minify them all in the correct order.

JSAN was a great idea that fell because it was focusing too much on the CPAN model. That and lack of time to devote to it.

[Nickolay Platonov]

Here are the results of my researches about dynamic loading problem:

http://extjs.com/learn/JScout
http://extjs.com/learn/Kanban

jScout is the basement of Symbie, and I consider it stable. It also  satisfies all mentioned requirements about external modules (even including libs, which are in turn uses their's own dynamic loading).
So I suggest to use it as codebase for Joose default loader.

[Nickolay Platonov]

I'm eager to post about the very 1st successful loading via mentioned scheme: see attachment.

The whole stressed test suite (about 100 files) was loaded with 2 http requests.

Currently implementation is quite dirty and needs to be sanitized, but the "proof of concept" is done!

[Malte Ubl]

Looks great! How come you are using post requests?

2009/2/19 Nickolay Platonov <nick...@gmail.com>:

[Nickolay Platonov]

I wasn't sure about - if there is a limit on size of http headers?
And just switched to POST to be safe.

[Malte Ubl]

Problem with Post is that it is not cacheable

[Nickolay Platonov]

I see.. Well, GET + header is generally the same, its easy to switch on it.
Will check the header's limit question.

[Nickolay Platonov]

Well, librarian reach the state, when the results should be reproducible on someone else machine )

To test it you'll need to have Catalyst installed:
cpan install Catalyst

After that get the sources from:
http://joose-js.googlecode.com/svn/branches/dependencies/

and to run the librarian with this line (from Joose-Library as current directory)
perl script/joose_library_server.pl -r -rd=1

The small web-server will start, and will show:
[info] Joose::Library powered by Catalyst 5.71000
You can connect to your server at http://localhost:3000

Open this url in browser:
http://localhost:3000/joose/tests/test.html

and check how the 14_modules_*.t.js test are executing in Firebug

Regards, Nickolay