archive_tw_resources enhancements and API-extension-proposal

[Diez B. Roggisch]

Hi all,

I'm currently working on a backwards-compatible extension to TW 0.9.6 that's
supposed to enhance the archive_tw_resources mechanism.

As many others, we are faced with the problem of too many requests per page.
And while the way TW lets you define widgets and declare dependencies between
them is really great for development, it creates a situation where you end up
with potentially dozens of script/css-files to load for each page.

To overcome this problem, a lot of sites just deliver an "all.js". Which of
course needs to be qualified with some id as well, to prevent caching-related
problems.

TW currently has no means of doing this - but the archive_tw_resources-command
is already there, and with a few lines of code I enhanced it to

- gather either js or css files
- pipe these potentially through the yui-compressor
- concatenate the result
- produce a hashed filename, and save the result under that name
- produce a mapping-file that declares which resources are part of the
aggregation

On the middleware-side of things, I'm adding a filter conceived as new
HostFramework that will inspect the injected resources, and if they are part
of the mapping, simply add the aggregated resource *once*.

So far, so good. However, there is one additional problem which I want to
tackle with a backwards-compatible API-extension.

A lot of 3rd-party-libraries (e.g. jquery and it's plugins) already ship with
various versions of their code - normal, debug, minified, packed.

Now if you declare all of these in say tw.jquery, the aggregation will end up
with several files with the (supposedly) same meaning inside.

To overcome this, I added something I call "partitions". I'm not to happy with
that name, any suggestion is welcome.

What changes is simply this: when declaring a resource, you can pass not only
a string for a filename, but also a dict.

This dict must have a key "normal", which is the default partition.
Additionally, you can declare other partitions, as you see fit. We should
aggree on a common set, such as debug, min, packed though.

now the resource-app is aware of these partitions - it can be configured to
deliver resources out of one of them, falling back to "normal" if there is
nothing found for the given partition.

An example:

b = JSLink(modname="tw",
filename=dict(normal="B.normal.js",
debug="B.debug.js")
).inject()

So with this scheme being utilized over your app (and of course hopefully
3rd-party-widget-packages), we have a situation where we can

- deliver various partitions while developing or on production
- if we want to go even further, create aggregated script-files that reduce
request-number and size

One issue shouldn't go unmentioned: dependencies. If e.g. your js depends on
jquery, you better make damn sure that it's included before your own Js.

But there is help: one can filter the aggregated files, and e.g. decide to let
the jquery-stuff become it's own aggregate. So you end up with something like
this in your site:

<script src="/jquery.min.js"/>
<script src="/jquery-plugins/50e1d1e9c8b1b556706ad876e22236-min.js"/>
<script src="/myapp/50e1d1e9c8b1b556706ad876e22236-min.js"/>

The above is just an illustration, the normal tw-middleware-prefixes would of
course appear.

I'm also not entirely sure if there isn't a way to overcome the dependency-way
at least to a certain extend.

So - all in all: what do you think? As I said, the changes are fully
backwards-compatible, so even if you chose to *not* use the aggregation and
partitioning, he doesn't lose anything.

Another topic I haven't given more than a cursory glance is the

http://code.google.com/apis/ajaxlibs/

facility - making that part of e.g. tw.jquery would also be nice.

Regards,

Diez

[Christoph Zwerschke]

Diez B. Roggisch schrieb:

> I'm currently working on a backwards-compatible extension to TW 0.9.6 that's
> supposed to enhance the archive_tw_resources mechanism.

Sounds good! How about simply "options" instead of "partitions"?

-- Christoph

[Diez B. Roggisch]

Christoph Zwerschke schrieb:

An option would be something that you can add. This is more of an mutual
exclusive thingy. I thought about "level", or maybe "distribution".

I need to have a better look at the dependency-mechanism, if I get a
better grasp on that, I hope I can create an all-encompassing solition.

I also gave some thoughts on "spriting", to reduce CSS-based latency.
But these are not condensed to code yet.

Diez

[Christoph Zwerschke]

Diez B. Roggisch schrieb:

> An option would be something that you can add. This is more of an mutual
> exclusive thingy. I thought about "level", or maybe "distribution".

How about "pack", "package" or "packaging"?

> I also gave some thoughts on "spriting", to reduce CSS-based latency.

You mean packing images together and displaying them with CSS clip?

-- Christoph

[Diez B. Roggisch]

Christoph Zwerschke schrieb:

> Diez B. Roggisch schrieb:
>> An option would be something that you can add. This is more of an mutual
>> exclusive thingy. I thought about "level", or maybe "distribution".
>
> How about "pack", "package" or "packaging"?

Sounds good. I'll ponder this a bit.

> > I also gave some thoughts on "spriting", to reduce CSS-based latency.
>
> You mean packing images together and displaying them with CSS clip?

Yes. It would sure be hard to do automatically, but OTOH I'd at least
like to give it a shot.

Diez

[Diez B. Roggisch]

Hi,

I'm pretty much done with the first implementation. I so far kep the
name "partition", as all others interfered with other concepts already used,
but I'd still think it should be renamed, so if there is any good
suggestion...

Now the question is: how do we go about putting this up for a test? I can push
it to the 0.9 main repository of course. Would that be ok?

The code is mostly tested, but it could need a bit more documentation and a
few test-cases more, I'll add these - but I'd wanted to ask first on how to
go about publishing this.

Diez