bootstrap a node with jclouds-chef and chef solo

[Pasquale Andriani]

I'm wondering if it is possible to bootstrap a node by using Chef Solo and not Chef Server. Is there any example? 

Cheers, 

P.

[Ignasi]

Hi Pasquale

Current version does not support Chef Solo. The bootstrap script that is generated configures chef-client and the first chef-client run, and assumes a Chef Server exists.

Aynway, supporting Chef Solo is a nice to have, so we can start a discussion about how to add support for it.

Ignasi

P.

--
You received this message because you are subscribed to the Google Groups "jclouds" group.
To view this discussion on the web visit https://groups.google.com/d/msg/jclouds/-/cdUXWsfWrAkJ.
To post to this group, send email to jcl...@googlegroups.com.
To unsubscribe from this group, send email to jclouds+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds?hl=en.

[Pasquale Andriani]

Can I add a line to Issues in google code?

[Ignasi]

Sure.

Since running chef-solo does not require access to a Chef Server, I think this could be addressed in a ChefSolo statement in the scriptbuilder project. The only thing you need to run chef-solo is an appropriate shell script to be executed once the node is running.

It should install chef-solo, create or download the desired cookbooks/recipes and create the solo configuration file. If you come up with a script that can do that, you have all you need to use chef-solo (you could run it using the runScriptOnNode method of the ComputeService). That script would be a great starting point to create the ChefSolo statement, so it can be easily reused and parametrized.

To view this discussion on the web visit https://groups.google.com/d/msg/jclouds/-/wbSGatU6xPoJ.

[Adrian Cole]

For chef, add it to the github issues list.

Thanks for piping up!
-A

To view this discussion on the web visit https://groups.google.com/d/msg/jclouds/-/wbSGatU6xPoJ.

[Adrian Cole]

+1 good idea, ignasi.

[Ignasi]

I've opened an issue in jclouds-chef (even if we end up implementing it in jclouds/scriptbuilder) [1].

Regarding implementation, chef-solo needs the cookbooks to exist in some place in the node in order to execute their recipes, so we should upload the cookbooks to the node before running chef-solo. There are two ways of doing this:

  * Invoke chef-solo with the "-r" parameter [2], where the value of this parameter is an URL pointing to a tarball with the cookbooks. Chef-solo will automatically download the tarball and extract its contents to the configured cookbook directory.

  * Manually upload the cookbooks to the node using (for example) the SSH put method of the jclouds ssh client, before running chef-solo. Once uploaded chef-solo could be invoked with the same "-r" parameter pointing to the tarball in the file system, and the same script would also work.

The thing to take into account here is that ChefSolo will assume that the tarball given in the "-r" option exists, and it is up to the user to configure it properly or upload the tarball to the node first.

If this approach is ok, we could implement the ChefSolo statement this way and make it accept the following input parameters:

  cookbook-url: local or remote url of the cookbook tarball.

  recipes: the list of the recipes to install.

  attribute-map: an attribute map to override the default attributes of the selected recipes

Thoughts?

[1] https://github.com/jclouds/jclouds-chef/issues/28

[2] http://wiki.opscode.com/display/chef/Chef+Solo#ChefSolo-RunningfromaURL

[Ignasi]

Hi Pasquale,

I've just added basic support for Chef Solo. You can take a look at this pull request to keep up to date: https://github.com/jclouds/jclouds/pull/950

This first implementation contains the topics commented in the previous email except the attributes support.

jclouds-devs: To support attributes and other ChefSolo stuff, the statement needs to manipulate JSON data. Which is the best way to populate a Json instance to the scriptbuilder Statement, in order to use it to generate json content?

Ignasi

[Pasquale Andriani]

Thanks a lot Ignasi! 

I just found the time to try it.

Unfortunately I'm getting some error in InstallRuby statement execution script [2]..I'm working on Ubuntu 12.04 and it seems that libruby-extras and libruby1.8-extras are no more available as in [1]..

Any known workaround? Are these packages really needed?

Best, 

P.

[1] http://www.ubuntuupdates.org/package/core/precise/universe/base/libruby1.8-extras

[2] https://github.com/jclouds/jclouds/blob/master/scriptbuilder/src/main/resources/functions/installRuby.sh#L5

[Adrian Cole]

On Nov 3, 2012 3:49 PM, "Ignasi" <ignasi....@gmail.com> wrote:
>
> Hi Pasquale,
>
> I've just added basic support for Chef Solo. You can take a look at this pull request to keep up to date: https://github.com/jclouds/jclouds/pull/950
>
> This first implementation contains the topics commented in the previous email except the attributes support.
>
>
> jclouds-devs: To support attributes and other ChefSolo stuff, the statement needs to manipulate JSON data. Which is the best way to populate a Json instance to the scriptbuilder Statement, in order to use it to generate json content?

Umm I might try to do this with string manipulation, if it is only composing on top-level jsonball from several peers.  Otherwise, it becomes tough to do correctly unless you use a builder pattern which ends with build(Json json) that can be obtained via context.utils()

[Ignasi]

Pasquale, thanks for trying! I tested the script in Centor 5 and Ubuntu 11.10 and it worked fine. I also found a pull request in chef [1] that fixes what you are experiencing, so I'll give it a try. They just install those packages optionally, so they may not be required (the InstallRuby statement is also used by the jclouds-chef project, so it is worth it to get it working with 12.04!)

You may also take a look at this pull request [2], where I am adding support for the entire list of options and configuration files supported by Chef Solo.

Anyway, I'll include the fix for the lib ruby-extra packages in a separate pull request, so stay tuned!

Ignasi

P.S. Adrian, in the last pull request, I implemented it with string manipulation. It is up to the user to build the appropriate json String, since in most cases may be quite trivial ones.

[1] https://github.com/opscode/chef/pull/263/files

[2] https://github.com/jclouds/jclouds/pull/968

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.
To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.

[Ignasi]

Hi again Pasquale!

I've just opened this PR [1] which removes the installation of the libruby-extras package. I've tested it in Ubuntu 11.10 and Centos 5 and it worked fine for Chef Solo and for the jclouds-chef statement that uses a Chef Server, so definitely those packages are not required to run Chef.

Could you give it a try with your Ubuntu 12.04 template?

Cheers,

Ignasi

[1] https://github.com/jclouds/jclouds/pull/980

[Pasquale Andriani]

Sure! I'll wait for the commit to be merged and let you know. Thanks a lot! 

[Ignasi]

Thx!

You can also clone my fork and build the fix-ruby-install branch to test it you don't want to wait.

To view this discussion on the web visit https://groups.google.com/d/msg/jclouds/-/m57mqnUazPgJ.

[Adrian Cole]

Merged.

Thanks for all the attention to this, guys.

-A

To view this discussion on the web visit https://groups.google.com/d/msg/jclouds/-/m57mqnUazPgJ.

[Pasquale Andriani]

InstallChefGems() statement works well also on Ubuntu 12.04!

Thanks a lot guys!

P.

[Adrian Cole]

Pasquale,

sweet! do you mind making an example on jclouds-examples github? I
think many people could be interested in this. You can use
compute-basics as a template for find/replace/change.

https://github.com/jclouds/jclouds-examples

-A

On Sun, Nov 11, 2012 at 9:04 AM, Pasquale Andriani

> https://groups.google.com/d/msg/jclouds/-/_7pt1K6U7s8J.

[Ignasi]

Just blogged some basic usage examples and how to combine Solo with Git to keep the nodes up to date:

http://blog.abiquolabs.com/cooking-your-own-configuration-in-the-cloud/

[Chris Strand]

Thanks Ignasi, that looks like a great write up - bookmarked!

 

Chris

[Adrian Cole]

Well done, Ignasi!

-A

[Pasquale Andriani]

Good job Ignasi! May I create the example by mentioning your post? I will do it soon as Adrian suggested.

Thanks, 

P.

[Ignasi]

Hi!

I have a work in progress version of the examples at [1]. Currently the Solo example is working, but the Chef Server based one is failing due to issues with the private keys (I hope to have it fixed by the end of this week).

Anyway, if you have some examples to push, the more examples we have the better! And if you can try the my Solo example, it would be very nice too :)

Thx for your dedication on this!

Ignasi

[1] https://github.com/nacx/jclouds-examples/tree/chef-examples/chef-basics

To view this discussion on the web visit https://groups.google.com/d/msg/jclouds/-/nEV0BZNFb2kJ.

[Pasquale Andriani]

Well, I can see that the one on which you are working is already quite complete!

Thanks, 

P.

[Ignasi]

I fixed the issues with the Chef Server and now the examples [1] are working for Solo and Standard Chef.

The example is a copy of the compute-basics stuff but replacing the 'exec' and 'run' commands by 'solo' and 'chef', to bootstrap the deployed nodes accordingly and install (by default) the Apache2 recipe.

Any testing and feedback is appreciated!

Ignasi

[1] https://github.com/jclouds/jclouds-examples/pull/19

[Adrian Cole]

Whoah. Cool, so this works both online and also solo!  Impressive demo, man.

-A