Improving the Cast client command line interface

[Tomaž Muraus]

One of our goals for the future release is to make the Cast client more easy to use.

Basically user should be able to deploy a new application and later on upgrade it with as few commands as possible.

To tackle this problem we should probably look at the Cast use cases and find the best command or combination of commands for a single use case.

Here is a list of some common use cases:

1. Deploying a first version of an application
1. manifest file doesn't exist yet
2. manifest file already exists
2. Upgrading to a new version of an application
3. Stoping, starting and restarting a service
4. Deleting / destroying an application instance
5. Viewing / tailing application (service) log file

Feedback is welcome.

Thanks,

Tomaz

[Tomaž Muraus]

OK, I think this so called "magic" command should look and work something like this.

Command usage

cast deploy [--app-path=<path to the application directory>] [--no-interactive]

Command options

• --app-path=<path to the application directory> - path to the directory where the application code and cast.json file is located (defaults to the current working directory)
• --no-interactive - don't interact with the user, immediately exit upon error with a non-zero exit code (this option will most likely be used when calling this command from the VCS's post receive hooks and other scripts)

Command actions in different contexts

cast.json file doesn't exist in the specified directory 

Print an error message and ask the user if he want to create a new manifest file (calls cast init internally). If the user answers "no", exit, otherwise create a manifest file and continue with the deployment process.

cast.json file exists, instance with the version specified in the cast.json file doesn't exist on the server and there are no other versions for this application already deployed on the server - this is a first version

1. Create and upload a bundle (if it doesn't already exist locally and on the server, calls cast bundles create and cast bundles upload internally)
2. Create a new instance with the following name: instanceName = applicationName.replace(/\s/, '_').toLowerCase();
3. Enable and start the new instance
4. Print a success message - "Application Foo Bar 1.0.0 has been successfully deployed. If the application did not start properly, you can use the following command to view it's log file: cast services tail instanceName"

cast.json file exists, instance with the version specified in the cast.json file already exist on the server

Print an error message - "App Foo Bar 1.0.0 already exists on the server. If you want to deploy a new version of your application, you need to change the "version" attribute in the cast.json file."

As a bonus we could maybe also add a new command for increasing / decreasing the application version number. For example: cast bundles increase-version, cast bundles decrease-version. This commands would of course only work if user is using a classic major.minor.patch versioning scheme

cast.json file exists, instance with the version specified in the cast.json file doesn't exist on the server, but there is already an existing version of this application deployed on the server

1.  Create and upload a bundle (if it doesn't already exist locally and on the server)
   
2. Upgrade the currently deployed version to the one which has just been uploaded (calls cast instances upgrade internally)
3. Print a success message "App Foo Bar 1.1.0 has been successfully deployed (version 1.0.1 has been upgraded). If the application did not start properly, you can use the following command to view it's log file: cast services tail instanceName"

For all this to work, we would need to change the way the user specifies an application version. Currently the version is manually passed to the bundles command (cast bundles create <name> <version>, etc.), but for this to work, a version would need to be specified in the manifest file.

I think specifying version in the manifest file has multiple advantages and I have already proposed this in the past:

• it's more explicit
• it's easier to track version-to-revision mapping if you are using some kind of version control system
• requires less interaction with the user when using "magic" commands like this one

I need to point out that this command is primary there to help users get up and running as soon and with as few steps as possible. If a user has special needs and / or wants a more fine-grained control, all the existing commands will still be available.

Feedback?

Thanks

[russell_h]

I like the direction this is going, but I have a few concerns:

1. Not a big one, but we need a --remote option
   
2. I think its a mistake to try to infer which instance to operate on (with the possible exception of when there is only one existing instance of the application)
   
3. As far as versions, I'm not too picky about where we get them from, but one thing thats been bothering me is that in our current model the only time a version is specified is when a bundle is created client-side. If Client A and Client B both create version "1.5" we have no way of distinguishing these bundles, knowing which is the "correct" version, or knowing who created them. I'm not sure if this is really a problem we need to solve, or if it is how we should do it, but we should maybe consider using a checksum of the tarballs or something?
   

Anyway, I think the command should look something like:

cast deploy [--app-path=<path to the application directory>] [--no-interactive] [--remote=<remote-name>] <instance-name> [<instance-version>]

cast.json doesn't exist

Create it with "cast init" and proceed

instance with given name doesn't exist

Create the instance using the given name and the current version, or the specified version if one was specified. Create and upload a bundle if necessary. Start the instance's service.

instance with given name does exist

Attempt to upgrade/downgrade (I'm not sure if its important to distinguish between the two) to the current or specified version and restart the instance service. Create and upload a bundle if necessary. Obviously this errors out if the existing instance is a different application or something. Ideally this should be idempotent, unless someone can think of a good reason for it not to be.

....

In short, any time someone runs the command its should attempt to ensure that the specified instance exits with the current or specified version. I think our ideal workflow looks something like this:

$ <working - create the initial version of the application>

$ cast deploy staging

$ <working - fix some bugs, etc>

$ cast deploy staging

$ <working - more fixes>

$ cast deploy staging

$ cast deploy prod

The one thing missing here is how versions get specified - Like I say, I haven't really made up my mind on this.

Thoughts?

[russell_h]

Also, I wasn't very clear about this:

With my proposal, specifying a version number to the "deploy" command doesn't create a bundle for that version unless it matches whatever is in the manifest, it would only look for a previously created bundle with that version. This would be mostly for downgrades (or upgrades following a downgrade, etc).

So, for example if you deploy "1.0" then "1.5", you can then run "cast bundles deploy 1.0" to downgrade. But you can't run "cast bundles deploy 1.6" to just arbitrarily name the current version 1.6 - you have to set it in the manifest file (or tag it in version control, if we wanted to do some sort of integration there I guess) first.

Let me know if this doesn't make sense...

[Tomaž Muraus]

Replies are in-line.

On Wed, Jun 1, 2011 at 7:28 PM, russell_h <russell...@gmail.com> wrote:

I like the direction this is going, but I have a few concerns:

1. Not a big one, but we need a --remote option
   

Yeah, I agree. The user should be able to specify a remote and if not specified, the default remote is used.

 

1. I think its a mistake to try to infer which instance to operate on (with the possible exception of when there is only one existing instance of the application)
   

Yeah, this makes sense. Maybe we should still infer the instance name from the application name if the instance name is not passed to the client? Or do we want to be explicit? I guess we need to find a good balance between being explicit and easy to use at the same time :)

1. As far as versions, I'm not too picky about where we get them from, but one thing thats been bothering me is that in our current model the only time a version is specified is when a bundle is created client-side. If Client A and Client B both create version "1.5" we have no way of distinguishing these bundles, knowing which is the "correct" version, or knowing who created them. I'm not sure if this is really a problem we need to solve, or if it is how we should do it, but we should maybe consider using a checksum of the tarballs or something?
   

True, but this is a client side conflict and we can't really do much here. The bundle which is uploaded first wins and when you try to upload a bundle with the same version which is already on the server it throws an error, right?

 

1. Anyway, I think the command should look something like:

cast deploy [--app-path=<path to the application directory>] [--no-interactive] [--remote=<remote-name>] <instance-name> [<instance-version>]

cast.json doesn't exist

Create it with "cast init" and proceed

instance with given name doesn't exist

Create the instance using the given name and the current version, or the specified version if one was specified. Create and upload a bundle if necessary. Start the instance's service.

instance with given name does exist

Attempt to upgrade/downgrade (I'm not sure if its important to distinguish between the two) to the current or specified version and restart the instance service. Create and upload a bundle if necessary. Obviously this errors out if the existing instance is a different application or something. Ideally this should be idempotent, unless someone can think of a good reason for it not to be.

....

In short, any time someone runs the command its should attempt to ensure that the specified instance exits with the current or specified version. I think our ideal workflow looks something like this:

$ <working - create the initial version of the application>

$ cast deploy staging

$ <working - fix some bugs, etc>

$ cast deploy staging

$ <working - more fixes>

$ cast deploy staging

$ cast deploy prod

Yep, looks OK.

 

The one thing missing here is how versions get specified - Like I say, I haven't really made up my mind on this.

I still think the version should be specified in the manifest, definitely not in both places because this just makes it more confusing.

Thoughts?

[Tomaž Muraus]

Ah, now I see what you mean. It kinda makes sense, but for this to work we would need to force the user to use a specific versioning scheme, otherwise the agent wouldn't really know if it should upgrade or downgrade it.

I think downgrading is not that common as upgrading and when we do support it, we should have a separate command for it - cast instances downgrade <instance name> <version to downgrade to>

Among the thing I have mentioned above, passing the version to this command and also having it specified in the manifest makes it confusing imo.

[Tomaž Muraus]

First prototype is up (https://github.com/cloudkick/cast/commit/623a6c119cd2e692bd7778b28ef6dd1cfbb23c08), the general work-flow would now look something like this:

$ cast deploy --instance-name foobar - create a new bundle, upload it, create a new instance and start it

<modify foobar application code, bump version attribute in the cast.json>

$ cast deploy --instance-name foobar - upgrades instance "foobar" to the new version defined in cast.json

$ cast deploy --instance-name barfoo - create a new "barfoo" instance based on the version defined in cast.json and start it

The only thing we still need to decide is if this command should support the "version" option. If we won't support it this means this command can only operate on a local application because it relies on reading the "version" attribute from the manifest file.

In this case it think it kinda makes sense to add "--version" option. It might make the command a bit more confusing, but as long as we document it well this shouldn't be problematic and it will make this command even more powerful and useful.

In any case, I will polish it up and add tests later today.

[Tomaž Muraus]

Actually, the more I think about the version attribute the less I like it :)

Version attribute would only tell the deploy command on which bundle / application version to operate on. First it would check if this version exists remotely, then locally in .cast_project/tmp/.

If the version exists either locally or remotely it would use this one, otherwise it would print an error. This is all fine and well, but the problem is that the user is still limited to the application which exists locally, because the "application name" attribute is read from the manifest file as well.

Not sure if we should also add "app-name" attribute to this command. We should probably keep this command simple and the users should use special commands for custom work-flows.

Comments?

[Tomaž Muraus]

I have added tests and merged this functionality into develop branch.

It is possible that the internals will still change a bit, but the user facing interface should stay the same.

Some feedback regarding the version argument would still be appreciated though.