Jclouds CLI

[Ioannis Canellos]

Hi,

every now and then people are asking about a jclouds cli and the usual response is to point people to jclouds-karaf. 

However, people not familiar with karaf will still have to download karaf, servicemix etc and install the jclouds shell commands there.

I was thinking about adding a small distribution under jclouds-karaf that will have everything there:

• jclouds preinstalled
• shell commands preinstalled
• a default set of providers preinstalled (?)
• jclouds branding
• reduced size

So people will just donwload jclouds-karaf and will be ready to use the shell.

Thoughts?

--
Ioannis Canellos

FuseSource

Blog: http://iocanel.blogspot.com

Twitter: iocanel

[Andrew Phillips]

> So people will just donwload jclouds-karaf and will be ready to use the
> shell.
>
> Thoughts?

Some initial thoughts:

* How hard is it to download the various components/how long does it take?
* Would it help to simply provide documentation on how to set this up?
* Could we provide an install script/Puppet manifest/whatever to do
this for you?

ap

[Ioannis Canellos]

Hi Andrew,

thanks for sharing your thoughts. The installation process is quite easy and we've already have some documentation around it.

Even if we provide scripts etc and make it super easy, people will still have to download a container that they probably never heard of and run some installation scripts.

My idea was to hide all the karaf bits, by providing a distro tailormade to jclouds, by removing all the karaf stuff and just keeping the:

jclouds shell commands

commands to add remvoe apis and provides

commands for configuring the services and that's all.

So a user would just have to download a distro we provide and just use the shell.

[Chris Custine]

+1  We should definitely do this.  Adrian and I have chatted about doing it but it has always been just out of reach on the priority list.  I would be tempted to have all providers and optional bundles packaged in the distro, but as you said, only have a few (aws, openstack, vcloud, etc) installed be default.

-- 

Chris Custine

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.
To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.

[Andrew Bayer]

Strong +1 - better docs on how to use jclouds-karaf would be nice too. I'm not clear on whether it's possible to, say, use jclouds-karaf from shell scripts, etc.

A.

[Ioannis Canellos]

Strong +1 - better docs on how to use jclouds-karaf would be nice too. I'm not clear on whether it's possible to, say, use jclouds-karaf from shell scripts, etc.

Well, I think that its easier to point people to a distro which has everything preinstalled. Most important is the fact that the distro can be tailor-made around jclouds and won't have all the karaf bits that might get users confused.

 

[Andrew Bayer]

Most definitely!

A.

[Andrew Phillips]

> Well, I think that its easier to point people to a distro which has
> everything preinstalled. Most important is the fact that the distro can be
> tailor-made around jclouds and won't have all the karaf bits that might get
> users confused.

I see your point but am wondering how much work maintaining a custom
distro woul d be vs. providing a (preferably executable, i.e. a script
or so) "recipe" that would make it easy for people to tweak if
desired, and would probably also be smaller and easier to update.

Well, at least that seems to be the general case: you certainly know
much more about Karaf and the relative complexity of maintaining a
"distro-install" script vs. putting together the distro itself.

Note that the "distro-install" script could be POM file, so may not be
that far away from a "distro build" in any case. I guess, though, that
a shell/batch file would be preferable in terms of placing fewer
requirements on the target system.

ap

[Andrew Phillips]

> My idea was to hide all the karaf bits, by providing a distro tailormade to
> jclouds, by removing all the karaf stuff and just keeping the:
>
> jclouds shell commands
> commands to add remvoe apis and provides
> commands for configuring the services and that's all.
>
> So a user would just have to download a distro we provide and just use the
> shell.

As per a Skype conversation with Ioannis just now, which made things a
lot clearer for me (thanks, Ioannis!), using Maven to produce a
downloadable archive indeed seems the way to go.
Especially as Karaf offers some Maven plugins to make re-packaging
with features easy, and there wouldn't be a need to change the Karaf
version unless fixes were required.

github.com/jclouds/jclouds-cli, anyone? ;-)

ap

[Chris Custine]

+1 I was thinking the same thing.

 

ap

[Adrian Cole]

done.

added you guys as collaborators to: https://github.com/jclouds/jclouds-cli

[Ioannis Canellos]

I have something that is usable, I'll push it shortly!

[Adrian Cole]

Hi, guys.

Thinking about this, I think we need to consider 2 ways of interacting.

1.  user wants to execute commands in a natural way from shell scripts

2.  user wants an interactive shell

cli is for unix users, shell is for power users.  An example of this is Chef's Knife[1] vs Shef[2]

CLI:

The first shouldn't require any karaf knowledge at all.  If there's any weird stuff or impl leakage, it is likely to fail to attract sysadmins or the like.  Many of these cloud administrators will be familiar with euca2ools[3], ec2 tools[4], or things like git[5]

$ ec2-describe-availability-zones --region us-east-1
AVAILABILITYZONE    us-east-1b  available   us-east-1   
AVAILABILITYZONE    us-east-1c  available   us-east-1   
AVAILABILITYZONE    us-east-1d  available   us-east-1

We should underscore the importance of being unix admin natural in a CLI, as otherwise antibodies will definitely arise!  I think our CLI approach needs to be validated with a java non-lover, to ensure we meet that goal.

SHELL:

Shell is for power users, and I see Karaf as a good option for this.  The shell could also implement the CLI, so long as it doesn't make things weird.

Make sense?

-A

[1] http://wiki.opscode.com/display/chef/Knife
[2] http://wiki.opscode.com/display/chef/Shef

[3] http://wiki.debian.org/euca2ools

[4] http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/OperationList-cmd.html

[5] https://github.com/dain/git-like-cli

> jclouds-dev+unsubscribe@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/jclouds-dev?hl=en.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "jclouds-dev" group.
> To post to this group, send email to jclou...@googlegroups.com.
> To unsubscribe from this group, send email to

> jclouds-dev+unsubscribe@googlegroups.com.

[Andrew Phillips]

> I have something that is usable, I'll push it shortly!

Looks cool! So you think that

https://github.com/jclouds/jclouds-cli/blob/master/assembly/pom.xml

was shorter than a shell script, hm..? ;-))

ap

[Andrew Phillips]

> done.

Cool, thanks!

ap

[Andrew Phillips]

Ah, interesting point! I had pictured this (simply from the demos of
jclouds-karaf I remember) as being mainly an *interactive* CLI/shell.

@Ioannis/Chris: was the intention also to use this to essentially
provide a suite of command-line tools, with all the things like
command redirection etc. that would be expected?

ap

[Ioannis Canellos]

What I initially had in mind was an minimal interactive shell, that would not require any knowledge of Karaf, yet it would provide all the cool code completion stuff. What is currently on github is really close to what I had in mind.

Adrian mentioned that it would be cool if the commands could be used directly from the shell without having to go in "interactive mode". That's seems really easy to do. So we can combined the two approaches and provide a complete solution.

@Andrew: I don't know if its shorter in terms of size, it definitely seems (to me) shorter in terms of effort and management overhead. I may be wrong, maybe one with better scripting skills than mine could make it easier :-)

[Andrew Bayer]

A knife-esque CLI for calling from shell scripts would be fantastic - otherwise, I'll end up writing my own. =) I'm a Java person, but I'm also a general build/scripting/etc guy, so I can give a pretty good sense of how the CLI fits into a scripting environment. Also I'm happy to start using it pretty much the moment it's ready for initial testing. =)

A.

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.

To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.

[Adrian Cole]

Andrew,

cool. so you are now the Customer :)

Which flags + command groups/actions do you need?

ex. for starters:

--provider
--identity
--credential
--endpoint

group: node
actions:
list
create
create-group
destroy
destroy-group
exec (execute a shell command)

[Ioannis Canellos]

Which flags + command groups/actions do you need?

ex. for starters:

--provider
--identity
--credential
--endpoint

group: node
 actions:
   list
   create
   create-group
   destroy
   destroy-group
   exec (execute a shell command)

To make sure we are all on the same track. so you imagine somethine like this:

jclouds --proivder aws-ec2 --indentity ???? --creadential ???? group mygroup list.

jclouds --proivder aws-ec2 --indentity ???? --creadential ???? node create mygroup

jclouds --proivder aws-ec2 --indentity ???? --creadential ???? group exec /path/to/my/script

??? 

[Adrian Cole]

yeah.. sounds about right..

jclouds --proivder aws-ec2 --indentity ???? --creadential ???? node

create mygroup 1

jclouds --proivder aws-ec2 --indentity ???? --creadential ???? group

list mygroup

jclouds --proivder aws-ec2 --indentity ???? --creadential ???? group
exec /path/to/my/script

AndrewB?

-A

[Andrew Bayer]

Make sure "provider" can be an API too, obviously. And I'm assuming "spin up X instances from group Y" is implicit. =)

A.

[Andrew Phillips]

> jclouds --proivder aws-ec2 --indentity ???? --creadential ???? group
> exec /path/to/my/script

Perhaps even

jclouds --provider aws-ec2 --authFile
my-identity-and-credential.properties node create ..?

To ensure credentials aren't exposed in the process overview and/or
command history?

ap

[Ioannis Canellos]

Ok, I have the basic commands also work without the interactive shell. In a manner similar to what has been discussed.

e.g.: jclouds node create --provider X --identiy X --credential X --os-family ubuntu --os-version 10.04 --adminAccess myGroup 10.

To get it working you'll need to build jclouds-karaf 1.5.0-SNAPSHOT first and then give it a shoot.

It's still needs quite some work as it has its glitches.

On windows currently only the interactive shell works. If anyone can volunteer to get the jclouds script working on windows it would be awsome.

[Adrian Cole]

Andrew.. how about a 1password extension :)

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.

To unsubscribe from this group, send email to jclouds-dev+unsubscribe@googlegroups.com.

[Adrian Cole]

Ioannis,

Cool!  I'll give it a try shortly

-A

[Andrew Phillips]

> Andrew.. how about a 1password extension :)

Teehee. A mind-reading extension would be even safer, I feel...well, a
mind-reader that could *only* read the credential in question,
obviously ;-)

ap

[Ioannis Canellos]

If any of you had the time to test drive the cli, would probably have found out that when its used in non interactive mode it has a glitch which prevents the shell to return for quite some time after the command execution. So I added a custom shell main class (a modified version of the one provided from Karaf) to work around it. Enjoy.

[Adrian Cole]

Thanks!  Will try it out, Ioannis.

-A

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.

To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.

[Paul Baclace]

Can someone tell me how to build and run jclouds-cli? I'm not a maven
expert, so what is the proper incantation and exactly how should it be
run without workspace edits and chmod which entail merges in later pulls?

(If only mvn had -projecthelp and a default target.)


Paul

[Andrew Phillips]

Hi Paul

Sorry to hear this is proving to be more tricky than hoped for. Here's
a command sequence that works for me (see env info below [1]):

> git clone g...@github.com:jclouds/jclouds-cli
> cd jclouds-cli
> mvn clean package
...(downloads the internet)...
> cd assembly/target

In there you should find a "jclouds-cli-1.5.0-SNAPSHOT.tar.gz" file
for Unix and a "jclouds-cli-1.5.0-SNAPSHOT.zip" for Windows.

I then unzipped the "jclouds-cli-1.5.0-SNAPSHOT.zip" file (dealing
with some duplicate JAR in the package) and was able to run the
bin\jclouds-cli.bat file fine:

jclouds> jclouds:provider-info --help
DESCRIPTION
jclouds:provider-info



SYNTAX
jclouds:provider-info [options]

OPTIONS
--credential
The credential to use for a compute service.
--endpoint
The endpoint to use for a compute service.
--api
The api or use.
--help
Display this help message
--identity
The identity to use for creating a compute service.
--provider
The provider or use.

If this doesn't work for you please indicate which command fails and
what the error is.

Thanks!

ap

[1] Output of "mvn -v"

Apache Maven 3.0.4 (r1232337; 2012-01-17 03:44:56-0500)
Maven home: C:\Program Files (x86)\Apache\maven-3.0.4\bin\..
Java version: 1.7.0_01, vendor: Oracle Corporation
Java home: C:\Program Files\Java\jdk1.7.0_01\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"

[Paul Baclace]

Ah, the missing steps were "cd assembly/target" and then expand one of
the tarballs. I filed a doc bug.

Thanks!

Paul

[Cosmin Dumitrache]

Hi all,

I was just trying the CLI today by following the instructions here [1].  I'm able to run simple commands like "jclouds" with no arguments and get the help message.

However, I'm not sure why I can not run any commands against AWS.

This is what happens with the CLI:

pegasus:Desktop cosmin$ jclouds node list --provider aws-ec2 --identity <my id> --credential <my credential>

[ . . . hangs forever . . . ]

This is what happens with the interactive shell.

jclouds> jclouds:node-list --provider aws-ec2 --identity <my id> --credential <my credential>

java/util/Collections

I also tried the "create node" command and I get the same behaviour. 

Any help would be appreciated.

Thanks,

Cosmin

[1] https://github.com/jclouds/jclouds-cli/blob/master/README.md

[Cosmin Dumitrache]

I made a pull request that fixes this [1]. Credits go to Aled.

Cosmin

[1] https://github.com/jclouds/jclouds-cli/pull/13

[Adrian Cole]

nice.

-A