Maestro example webapp (cookbook)

[Matt]

Hi Brian,

Maestro looks very promising, I am going to try and implement in my
webapp to deploy to EC2.

I just wondered if you have a working configuration (rails webapp
example) with chef cookbooks and recipes for nginx + passenger + rvm +
(rails3, rds, elb which you have given example of) or anything close
to help me get started.

I have been playing with Chef (client + opscode server platform) to
deploy a webserver similar than above and it's quite an exercise. !
So far i've been able to setup up rvm, ruby and gems. But i haven't
been able to get rails install through chef.

So any help would be appreciated.

Thank you / Matt

[Brian Ploetz]

Hey Matt,

Thanks for checking out Maestro!

I have a working app with the following:

nginx + passenger
MySQL running on EBS (per Eric Hammond's excellent tutorial here:
http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=100&externalID=1663)
postfix
monit

I don't have this set up hooked up to RVM yet though. I'm still
running it with Ruby 1.8.7 that I just pull down via apt. Shouldn't be
too hard to write a recipe for RVM (I'm sure there's an existing one
out there somewhere).

I will get this into a presentable form and post it up on GitHub as an
example Maestro app this weekend. I'll drop a note here when it's up
there.

Stay tuned........
BP

[Matt]

Ok, that's great, i'll keep an eye for it.

For RVM I have used this brilliant blog post:
http://www.agileweboperations.com/chef-rvm-ruby-enterprise-edition-as-default-ruby/

A recipe has been created in github here: http://github.com/denimboy/cookbooks/tree/master/rvm
So that should help to hook up rvm in maestro.

A question also with maestro:
- does it support the ability to scale the infrastructure when suppose
you want to add an extra node to a working and operating
configuration?
What maestro steps would you take to activate that node and get elb to
start handling traffic to that node?


I've got a suggestion for ELB (Load Balancer).
As well as (or instead of?) specifying the nodes names manually that
the ELB fronts, it would great if maestro could take a role (or role
list) so it could gather the nodes automatically from their role (from
the one defined in the ec2_node part).
This screencast by Adam Jacob (opscode) [http://blip.tv/file/2689674]
demonstrates the idea.
This way it facilitates maintenance of ec2_nodes, and help when
scaling up/down the number of nodes.

E.g.
elb_node "lb-1" do
availability_zones ["us-east-1b"]
listeners [{:load_balancer_port => 80, :instance_port =>
80, :protocol => "http"}]
ec2_roles ["web"]
health_check(:target => "TCP:80", :timeout => 15, :interval =>
60, :unhealthy_threshold => 5, :healthy_threshold => 3)
end

What do you think?

Looking forward to working with maestro. Well done for your work, it's
very nice.
It's a very nice alternative to poolparty or ec2_on_rails.

Cheers / Matt

On Oct 9, 1:02 am, Brian Ploetz <bplo...@gmail.com> wrote:
> Hey Matt,
>
> Thanks for checking out Maestro!
>
> I have a working app with the following:
>
> nginx + passenger

> MySQL running on EBS (per Eric Hammond's excellent tutorial here:http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=...)

[Brian Ploetz]

Hey Matt,

Follow ups below.....

On Oct 8, 3:25 pm, Matt <mloh...@gmail.com> wrote:
> A question also with maestro:
> - does it support the ability to scale the infrastructure when suppose
> you want to add an extra node to a working and operating
> configuration?
> What maestro steps would you take to activate that node and get elb to
> start handling traffic to that node?

I just took a quick peek at the code, and it doesn't look like it
currently supports adding a new node to a running ELB. It definitely
should. If you could file an Issue on the GitHub page I'll get that
supported. (Or, if you would like to do this yourself, pull requests
are always welcome!). Also note that along the same vein, Maestro
currently doesn't support auto-scaling. There's an existing Issue on
GitHub about this, I just haven't gotten around to it yet.

> I've got a suggestion for ELB (Load Balancer).
> As well as (or instead of?) specifying the nodes names manually that
> the ELB fronts, it would great if maestro could take a role (or role
> list) so it could gather the nodes automatically from their role (from
> the one defined in the ec2_node part).
> This screencast by Adam Jacob (opscode) [http://blip.tv/file/2689674]
> demonstrates the idea.
> This way it facilitates maintenance of ec2_nodes, and help when
> scaling up/down the number of nodes.
>
> E.g.
>     elb_node "lb-1" do
>       availability_zones ["us-east-1b"]
>       listeners [{:load_balancer_port => 80, :instance_port =>
> 80, :protocol => "http"}]
>       ec2_roles ["web"]
>       health_check(:target => "TCP:80", :timeout => 15, :interval =>
> 60, :unhealthy_threshold => 5, :healthy_threshold => 3)
>     end
>
> What do you think?

I think that's fine. The reason I started with specifying node names
instead of roles is that it's more fine grained control over the nodes
you want fronted by the load balancer. For example, you may have a
generic "web" role that configures your standard web stack, but you
might have different application types using that role (i.e. an
internal admin app on one node, and the public facing app on N other
nodes), and you wouldn't want all of these fronted by the load
balancer. Having said that, supporting role names as well is certainly
more flexible, it just puts the onus on you to "do the right thing".
If you can file an Issue on the GitHub page for this feature request,
I'll add it. Shouldn't be a big deal.

Thanks.
BP

[Matt]

Hey Brian,

I've submitted those 2 issues in github maestro's project.

I'll keep playing with it.

Cheers / Matt

[Brian Ploetz]

Matt,

I added an example Rails 3 app which uses Maestro. You can find it
here:

http://github.com/bploetz/maestro-rails3-example

It doesn't have everything you enumerate below, but it should be
enough to get you started......

BP

On Oct 8, 4:29 am, Matt <mloh...@gmail.com> wrote:

[Matt]

Hey Brian,
Yes I saw your other post, thank you for that. I'm trying it out.

What do you think I should do about Ruby on the EC2 instance? Install
manually ruby or through rvm?
Have you tried if your recipes work with rvm? Which ruby version did
you install?

I have tried to make maestro work with rvm. Initially I had rvm as a
recipe, but it would not work properly.
It would install rvm on ec2 but the chef-solo had already started in
the system's ruby version and would not run in rvm's ruby.
So i then switched to modify the install chef-solo script to install
rvm first, with ruby 1.9.2.
But then chef-solo would not run any recipes, main process would exit
after installing and configuring chef-solo.

I haven't been able to work out why that was the case yet. It's in my
fork (aquam8) in the ubuntu.rb file.

It would be good to get that working somehow. Best would be as a
recipe tho, and get chef-solo to run as rvm's ruby.
See if you have more luck than I.

Talk to you soon,
Thanks a lot / Matt

[Brian Ploetz]

Hey Matt,

Ideally you would manage Ruby VMs that you need for your app server
with RVM. You can do that through a Chef recipe (in theory). It
shouldn't matter to you as an end user which version of Ruby chef-solo
uses under the hood, but I have in fact planned to have Maestro use
RVM instead of installing Ruby via a package manager. I just created
an issue for that.

http://github.com/bploetz/maestro/issues#issue/17

BP