Alternate/Multi-Catalog Support

[Kris Kowal]

I just pushed this to my refactor branch. Here are the docs:

http://github.com/kriskowal/narwhal/blob/refactor/docs/catalog.md

The new "tusk update" replaces "tusk create-catalog" and adds support
for .tusk/sources.json, which can be composed of any number of
packages and can override the package descriptor sources for
particular packages.

Kris Kowal

[Christoph Dorn]

This looks great! I'll send more feedback when I have had a chance to
play with it.

WRT packageUrl in the package descriptors; and github package descriptor
sources:

The package being referenced may not sit at the root of the archive or
github repository. I would like to see some kind of *path* property to
be able to reference such packages deeper in the directory tree.

Christoph

[Kris Kowal]

On Sat, Dec 12, 2009 at 11:04 PM, Christoph Dorn
<christ...@christophdorn.com> wrote:
> WRT packageUrl in the package descriptors; and github package descriptor
> sources:
>
> The package being referenced may not sit at the root of the archive or
> github repository. I would like to see some kind of *path* property to
> be able to reference such packages deeper in the directory tree.

This is more involved that the process of constructing the packageUrl?
Would it be best to construct some sort of abstraction layer that
allows us to extract content from archives at particular locations, or
perhaps drilling into archives recursively? Perhaps you could clarify
the use case; I'm not sure how broad the scope of the solution ought
to be.

On a related note, I created a lib/http/fs-store module that we can
both use for caching content off the web in .tusk/http. You can grab
an HttpFsStore singleton from TUSK.getHttpStore() and then from there
convert URL's to Path objects. As I recall, you were using something
like that for using-packages, so we might get some reuse out of it.

Kris Kowal

[Bryan Berry]

awesome! I look forward to playing w/ this this week

--

You received this message because you are subscribed to the Google Groups "Narwhal and Jack" group.
To post to this group, send email to narw...@googlegroups.com.
To unsubscribe from this group, send email to narwhaljs+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/narwhaljs?hl=en.

[Christoph Dorn]

Kris Kowal wrote:
> On Sat, Dec 12, 2009 at 11:04 PM, Christoph Dorn
> <christ...@christophdorn.com> wrote:
>> WRT packageUrl in the package descriptors; and github package descriptor
>> sources:
>>
>> The package being referenced may not sit at the root of the archive or
>> github repository. I would like to see some kind of *path* property to
>> be able to reference such packages deeper in the directory tree.
>
> This is more involved that the process of constructing the packageUrl?
> Would it be best to construct some sort of abstraction layer that
> allows us to extract content from archives at particular locations, or
> perhaps drilling into archives recursively? Perhaps you could clarify
> the use case; I'm not sure how broad the scope of the solution ought
> to be.

I think just a path ought to do it.

e.g.

packageUrl = http://github.com/cadorn/narwhalrunner/zipball/master

references the archive file and

packagePath = packages/devtools

could reference the package I really want from the archive.

Same for github where ref identifies the branch, etc. and a *path*
property can reference the root of a package.

This feature is essential for my dev setup as I use a sea to construct a
dev environment where the programs I develop reside in program packages.
If I want to keep the whole sea in a repo I need a way to reference the
sub packages which are not at the root of the repo.

A simple *path* will suffice for referencing CommonJS compliant packages.

In the docs you mention that you can overwrite the package.json provided
by a package by specifying a descriptor in the github source descriptor.
This is a great feature to make modifications to an existing
package.json file or to provide one where it does not exist but requires
the package modules and to a lesser extent the layout to follow CommonJS
standards to be usable in a CommonJS system.

I have been thinking about how we can wrap existing non-CommonJS
compliant projects (or more applicably modules) to make them usable in
CommonJS systems. This would help to bridge the gap until the original
project provides a compatible distribution. Is this something you were
alluding to with "extract content from archives at particular locations"?

Third party projects could be wrapped by extracting the archive (if
applicable) and getting tusk to run a build module over it that can
re-arrange the content and patch files accordingly. This build module
could reside with the project (to provide an automated way to build a
CommonJS compliant codebase) or on a public URL directly or via a
package to enable others to use it if the original project does not
include it.

Over time I see a number of generic wrapper packages emerging that can
potentially allow wrapping of projects via configuration only.

To support this we could include the following properties in the
descriptors:

{
"packageUrl": "http://github.com/cadorn/narwhalrunner/zipball/master"
"packagePath": "packages/devtools"
"packageWrapper": {
// a package descriptor with an important property
"main": "wrapper"
}
}

The "main" gets called by tusk during the installation process and the
wrapper can include any additional resources to make the wrapping work.

The modifications required to get this working are minimal and could
allow us to pull in a lot of existing projects especially for use in
browsers. I for one want to be able to manage all resources used by my
sites via tusk even if the package modules are not CommonJS compliant.

> On a related note, I created a lib/http/fs-store module that we can
> both use for caching content off the web in .tusk/http. You can grab
> an HttpFsStore singleton from TUSK.getHttpStore() and then from there
> convert URL's to Path objects. As I recall, you were using something
> like that for using-packages, so we might get some reuse out of it.

I am planning on incorporating this module as I clean up my implementation.

Christoph