Cloud dependencies, task and instance roles
avishai
the intention of using it as a base for a rather complex deployment on
EC2, and it look like a fine peace of software.
Some of the features I'd like are currently missing and hopefully I
will be able to implement them. Most of these feature will probably be
poolparty plugins but some seem better suited to be core features. I
understood that some of the ideas i'm about to present have been
previously promoted but I would like to reignite the discussion so
whatever code I write will remain true to concepts of the project (and
also perhaps peek your brain in the process?).
So, here goes. The main features i believe should go into the codebase
are:
1. cloud dependencies - use case: app dreamweb consists of two clouds,
backend cloud with databases, filers etc. and app servers cloud,
webservers cloud. the clouds are dependent upon each other on the
application level and also on the system level (perhaps /etc/hosts
needs to be updated with the name of the mysql master)
On the applicative level, cloud dependencies mean that a depended
cloud requires every dependency to be satisfied - that is have at
least 1 instance running and responding to verify blocks. other nice
extensions may be configurable info properties. E.G.
cloud :webservers do
instances 1..10
using :ec2
depends_on_cloud :backend do
fw.open_for cloud.nodes
end
has_hosts_line "mysql_master #
{clouds.backend.mysql_master.hostname}"
clouds.backend.mysql_slaves.nodes.each {|node| has_hosts_line
"mysql_slave #{node}"}
end
cloud :backend do
using :ec2
instances 2
end
### END
the fw.open_for is just an example for a dependency block that is run
AFTER the backend cloud has started on backend nodes with info from
the current cloud.
2. tasks/instance roles - use case: cloud backend holds instances that
serve as mysql servers. most mysql setups use 1 master, 1 master
candidate (a singled out slave) and various slaves. while most of the
settings for the instances are the same there are settings that depend
on the actual task of the instance and of course the most important
info is the private IP address of the instance serving as master/
master candidate.
E.G.
cloud :backend do
using :ec2
#instances 1..4
task :mysqlmaster do
instances 1
promote :from => :mysqlcandidate, :how => "run some script or
maybe a chef recipe"
end
task :msyqlslave do
instances 0..4
end
task :mysqlcandidate do
instances 1
end
end
### END
This can be achieved with 3 different clouds but requires massive
config-fu (security groups must be the same) and we get the ability to
"promote" instances from one task to another (perhaps inheriting
elastic ip's etc.).
I would like your thought on the subject, don't be shy and feel free
to mock my syntax or curious ideas.
Avishai