volo experiment
James Burke
I'm playing around with a general bootstrap script that can pull code
down from github and use it to create new projects and download
scripts to add to the project:
https://github.com/volojs/volo
I purposely avoided trying to code up a registry system since that
seems to entail a bunch of work and I liked the idea of using github
as a type of registry. GitHub cannot be used for every type of code
fetching scenario, but it can handle a decent subset and it made it
easy to start.
The 'add' command does not trace nested dependencies yet because the
github pull code really requires 'owner/repo' IDs where most
dependencies are maybe just 'repo' or 'module ID'. I'm hoping I can
use some sort of github search api, but the v3 version does not seem
to have one:
https://github.com/volojs/volo/issues/7
Anyway, just sharing. I don't think volo solves all the needs
expressed on this list before, but it was a way for me to play with a
command line tool that talked to github to fetch code. I'll probably
continue to hack around with it for a while, but if you think it
should go in a particular direction or it could be merged with other
code or ideas give a holler.
I looked at the ssm code, but it seems to be focused on dealing with
registry/package.json-type of data at the moment, so I did not think I
could build on it yet. I also like the idea of delivering just one JS
file for people to use. That JS file can install other tasks, but I
think it helps if the starting point is just one file. Anyway, just
rambling.
James
Michael
Firstly, fair warning: I intend to exercise OSS license to reuse some of volo's features in sesame!
In general, however, I think that, architecturally speaking, it looks like volo is typical of most tools in this space, in that it wants to serve the kind of developer who is working alone or on a small team, and working on a small website. To be useful in my workplace there would need to be some additional goals added to the design.
1. Version-awareness. GitHub is indeed awesome, but where the concept of a package, and repo descriptors come into use is with long-running projects. For example if I always wanted to keep my installed foo.js package 1 version behind the current latest release version, I might need to do this:
volo latest foo
> 1.2.7
volo install foo v1.1
> installed foo v1.1.4
Or more simply...
volo updates
> foo can be updated from 1.1.4 to 1.2.7
> etc...
volo update
> installed foo 1.2.7
> etc...
Without the concept of a release that has meta data, like a release version identifier, none of that can be possible. Which is probably fine if your projects are short-lived enough that they won't require any convenient version management to be made available.
2. One file per install. This really is a weakness for an enterprise sort of environment, where optimisations around number and size of requests require fine control. However I see that such control can easily be applied after volo has done its work.
3. GitHub only. We might need to consider a company repo as higher priority, but fallback to GitHub if that is more appropriate. So some concept of configuring the tool to use a priority ordered list of repos, and some way of specifying which repo we mean is needed.
Are these issues that volo is concerned with? How would you foresee that they could be handled?
--
Michael
James Burke
> Hi James,
>
> Firstly, fair warning: I intend to exercise OSS license to reuse some of volo's features in sesame!
I hope so, one of the benefits of OSS!
> In general, however, I think that, architecturally speaking, it looks like volo is typical of most tools in this space, in that it wants to serve the kind of developer who is working alone or on a small team, and working on a small website. To be useful in my workplace there would need to be some additional goals added to the design.
>
> 1. Version-awareness. GitHub is indeed awesome, but where the concept of a package, and repo descriptors come into use is with long-running projects. For example if I always wanted to keep my installed foo.js package 1 version behind the current latest release version, I might need to do this:
>
> volo latest foo
>> 1.2.7
> volo install foo v1.1
>> installed foo v1.1.4
>
> Or more simply...
>
> volo updates
>> foo can be updated from 1.1.4 to 1.2.7
>> etc...
> volo update
>> installed foo 1.2.7
>> etc...
>
> Without the concept of a release that has meta data, like a release version identifier, none of that can be possible. Which is probably fine if your projects are short-lived enough that they won't require any convenient version management to be made available.
volo allows installation by version:
volo.js add documentcloud/bacbone/v1.1.4
The last path segment is a repo tag. By not including the version tag:
volo.js add documentcloud/bacbone
it will scan through the list of git tags that look like version tags,
order them and choose the latest one. If there are no version tags,
then it uses "master".
However volo.js does not currently have a command to list the version
tags it finds. I'll put that on the list -- it will be easy to do, the
data is already there, just need to dump it out.
> 2. One file per install. This really is a weakness for an enterprise sort of environment, where optimisations around number and size of requests require fine control. However I see that such control can easily be applied after volo has done its work.
I'm not sure I follow this one, can you expand on it? Do you mean that
volo only looks like it installs one file for a given repo (it can and
will install a directory of code)? Is this related to doing an
automatic concat of scripts after a build?
> 3. GitHub only. We might need to consider a company repo as higher priority, but fallback to GitHub if that is more appropriate. So some concept of configuring the tool to use a priority ordered list of repos, and some way of specifying which repo we mean is needed.
Agreed, I can definitely see the need for companies to have their own
internal repo. If that repo can support the parts of the GitHub API
that volo uses, then it could just slide in under the covers. volo has
a config object, which sets the location of the github to use, so that
could be pointed somewhere else.
The API that would need to be supported is fairly light too:
Get a list of tags:
/repos/{owner}/{repo}/tags
Raw API used to pull out just one file from the repo:
/{owner}/{repo}/blob/{tag}/{path/to/file.js}
Get a tar.gz of the repo at a given tag:
/{owner}/{repo}/tarball/{tag}
While GitHub could change its API in the future, I would not expect
that for a while, and I would expect it would be easy to
change/provide an internal adapter to volo if it ever were to change
and the repo you are hitting is github.com.
The big thing is having a search API that resolves something like
'backbone' to be 'documentcloud/backbone'. That is the next big issue
I want to tackle.
I did not want to have to start with designing a code registry or
trying to shoehorn in npm's registry. Although, volo is set up to be
able to resolve different types of archive names, so doing something
like:
volo.js add npm:backbone
would work once a volo/resolve/npm.js is created.
But mainly I saw GitHub as already providing the basic registry, and I
think there could be some more great leverage to using its API as the
default archive name format understood by a tool.
But I'm open to other thoughts too. Right now it is just a big experiment.
James