Best way to manage private packages?

[Daniel Pfile]

We have some business logic that has no reason to be in a public kanso package, but is shared among several of our node.js and kanso applications. Since packages can be both kanso and npm packages, we can pull the library into an npm package that lives in our private gitolite install, but there is no way to store a kanso module in git like that. The kanso options let you specify an array of repositories, but attempting to override that with 'kanso install --repositories' (in our Makefile for example) limits you to only one repository. Git submodules could work, but submodules in the packages directory and the associated .gitignore feels very hacky. So that leaves the only option as to replicate the entire http://kan.so/repository db, and add our specific packages to it. Is that the intended method? Is there a reason to not install packages with git like npm, other than "It not been written yet, patches welcome"?

Thanks!

-- Daniel

[Caolan McMahon]

It is possible to use multiple repositories, but you have to set them in
your .kansorc file. This can either be set for the current user
(in $HOME/.kansrc), or per-project by adding a .kansorc file in the
project's root directory.

You need to specify *all* repositories, including the kan.so one:

exports.repositories = [
'http://kan.so/repository',
'http://hostname/my/repository'
];

The order of repositories in this array is significant, with package
matches from earlier repositories taking precedence over later ones.

You should be able to use a new CouchDB database with no design doc to
publish packages to as a repository if you wish, but I'd recommend
downloading and pushing the repository app from
http://github.com/kanso/repository

Sorry about the Kanso-specific branding and navigation, I'll be
releasing a more generic build soon (and hopefully then I'll get around
to actually documenting this).

The other option is to just check your packages/* files directly into
git. We've been using this method on a few projects and it seems to work
reasonably well.

The ability to install package from git sounds interesting. How would
that look in kanso.json?

Let me know if you have any problems, and if you have time I'm sure
other Kanso developers will be interested in you documenting your setup
on the wiki ;)

- Caolan

[Daniel Pfile]

On Mon, Feb 27, 2012 at 12:32 PM, Caolan McMahon <caolan....@gmail.com> wrote:

It is possible to use multiple repositories, but you have to set them in
your .kansorc file. This can either be set for the current user
(in $HOME/.kansrc), or per-project by adding a .kansorc file in the
project's root directory.

You need to specify *all* repositories, including the kan.so one:

exports.repositories = [
   'http://kan.so/repository',
   'http://hostname/my/repository'
];

The order of repositories in this array is significant, with package
matches from earlier repositories taking precedence over later ones.

This should work OK, I did not realize I could have a .kansorc stored with the project. I have also written a simple wrapper to a locally installed kanso to extract config vars out of the node.js application and share them with kanso:

var kansorc = require('kanso/lib/kansorc');

var push = require('kanso/lib/commands/push');

var conf = require('../lib/getConfig');

kansorc.load(function (err, settings) {

  push.run(settings, ['http://'+conf.get('dbhost')+':'+conf.get('dbport')+'/'+conf.get('dbname')]);

});

I'm sure that's a totally unsanctioned use, but it works.

You should be able to use a new CouchDB database with no design doc to
publish packages to as a repository if you wish, but I'd recommend
downloading and pushing the repository app from
http://github.com/kanso/repository

Sorry about the Kanso-specific branding and navigation, I'll be
releasing a more generic build soon (and hopefully then I'll get around
to actually documenting this).

It doesn't look like it works very well when it's not mounted at /packages, did you map that url with a front end proxy or insecure rewrites?

The other option is to just check your packages/* files directly into
git. We've been using this method on a few projects and it seems to work
reasonably well.

I had thought about this, but didn't really like the implications, there was a blog post today that sums up some of the issues doing that with npm, and their solution:

http://blog.nodejs.org/2012/02/27/managing-node-js-dependencies-with-shrinkwrap/

 

The ability to install package from git sounds interesting. How would
that look in kanso.json?

I would think it would look just like npm:

Get the latest version from the configured repos:

"mypackage": null

 

Get a specific version from the configured repos:

"mypackage": "0.0.1"

Get master from a git+ssh repo:

"mypackage": "git+ssh://git@myhost/mypackage"

Get a tag from a git+https repo:

"mypackage": "git+https://pfi...@github.com/pfiled/mypackage.git#v0.0.1"

This is documented in npm help install.

Let me know if you have any problems, and if you have time I'm sure
other Kanso developers will be interested in you documenting your setup
on the wiki ;)

I will see what I can do with some wiki docs, perhaps a simple demo project combining kanso, npm and a Makefile. I'm interested in seeing what other patterns people use with node.js and couchdb. How do you configure proxies, caching, manage _changes responders, etc?

-- Daniel

- Caolan

[Caolan McMahon]

> I would think it would look just like npm:
>
> Get the latest version from the configured repos:
> "mypackage": null
>
> Get a specific version from the configured repos:
> "mypackage": "0.0.1"
>
> Get master from a git+ssh repo:
> "mypackage": "git+ssh://git@myhost/mypackage"
>
> Get a tag from a git+https repo:
> "mypackage": "git+https://pfi...@github.com/pfiled/mypackage.git#v0.0.1"
>
> This is documented in npm help install.

I'd be interested in pull requests to add this :) Failing that please
create an issue so I might have a chance of getting around to it myself.

Caolan

[Caolan McMahon]

The other option is to just check your packages/* files directly into
git. We've been using this method on a few projects and it seems to work
reasonably well.

I had thought about this, but didn't really like the implications, there was a blog post today that sums up some of the issues doing that with npm, and their solution:

http://blog.nodejs.org/2012/02/27/managing-node-js-dependencies-with-shrinkwrap/

The main concern here seems to be checking binaries in to a repository and the fact that it feels 'wrong' to some people. TBH I've had too many dependencies disappear from NPM to want to rely on a remote repository any more, so for most projects I check in my dependencies into git. Also, I don't know of any Kanso packages that include binaries and there are no special install scripts that can be run by a Kanso package at install time (as happens with NPM)... so it probably makes more sense here than it does with Node.

Still, I can understand people thinking it's messy. I just find it safer and easier when you need to debug a specific version of a project (including it's deployed dependencies).

[Milan Andric]

Related link:

http://www.mikealrogers.com/posts/nodemodules-in-git.html

--

Milan

[Daniel Pfile]

I have started on this in my spare time. Hopefully I can send you a pull request soon.

[Daniel Pfile]

Even more related links:

http://groups.google.com/group/npm-/browse_thread/thread/f12457843eae3269 

Some nice tips on how to ignore compiled deps.

Thanks for the feedback everyone.

-- Daniel

[Arthur McFlint]

On Monday, February 27, 2012 9:16:38 PM UTC+1, Caolan McMahon wrote:

> I would think it would look just like npm:
>
> Get the latest version from the configured repos:
> "mypackage": null
>
> Get a specific version from the configured repos:
> "mypackage": "0.0.1"
>
> Get master from a git+ssh repo:
> "mypackage": "git+ssh://git@myhost/mypackage"
>
> Get a tag from a git+https repo:

> "mypackage": "git+https://pfiled@github.com/pfiled/mypackage.git#v0.0.1"

>
> This is documented in npm help install.

I'd be interested in pull requests to add this :) Failing that please
create an issue so I might have a chance of getting around to it myself.

Hi,

that sounds like a great idea. Did anything happen along those lines since this post? If not, I may take it on as I'd like to use an internal repository.

Arthur