Building a better puppetrun and related ideas
Michael DeHaan
simple but enhanced puppetrun.
Basically the idea is merging the ideas behind Func and Puppetrun.
Obviously other tools like mcollective have various other advantaged
features so this will be fairly primative by comparison, though it
won't require a message bus. If you want something more advanced
obviously try out those tools, this is covering a much smaller use case.
This is something I am going to take a crack at this in the coming
weeks. This would be something pretty simple and lightweight, and
could
probably fix a lot of the use cases around making puppetrun (or
staggering large sets of hosts) a lot easier.
Features I'm thinking of:
Requires no additonal ports, setup, or config files -- use existing
puppet listening capability and puppetca, just a /usr/bin app
Be able to query dashboard DB to run against tagged nodes or hosts
that have certain data there (or in storeconfigs???)
Be able to run against wildcarded nodes based on what certs are
present on the puppetmaster (we know the hostnames)
Be able to be used easily from an API perspective from any ruby application
Be able to invoke ralsh remotely for querying things (and for debug,
and one off tasks)
Be able to run shell commands for things that are one offs (emergency
security power down now)
Example syntax:
punc --hosts *.example.org --puppetize # get new catalog and run
punc --hosts *.example.org --ralsh "service name=foo ensure=running"
# perform an action through ralsh
punc --hosts *.example.org --shell "/bin/emergency_script" # run a
shell script... for the one-off cases
punc --hosts foo.*.example.org --ralsh "service name=foo"
--format=json # query something with ralsh and generate a report
punc --hosts foo.*.example.org --facter fact --format=json # similarly
generate a facter report
punc --tags webservers [...ditto...]
punc --critiera "fact==foo" [..ditto...]
punc --critiera "fact==foo" [some operation to run only if fact
matches] [...ditto...]
So for example we could choose to reboot all the servers that match a
given fact, etc.
It should also allow easier staged deployments and environment usage
from apps that want to use the API.
Additional ideas for stuff you would like to see?
--Michael
Scott Smith
> Example syntax:
>
> punc
Python or Ruby?
-scott
Michael DeHaan
Totally Ruby :)
I think the RAL really makes up for and eliminates the need of a lot
of the stuff we tried to do as Func modules. Good stuff. I think
most people use Func to run bash regardless... however the main thing
we want to do here is just tell Puppet to wake up and run.
(Combining this workflow with no-op mode for instance)
Similarly, facter and everything is nicely extensible and can be
easily rigged up to do things like the "Exploding Laptop Finder" demo
script Func had ... I'd also like to do something similar to
https://fedorahosted.org/func/wiki/FuncInventory ... basically it's a
really quick way to get an RSS feed of an arbitrary anything.
--Michael
Daniel
normal puppetrun? Something like run a shell command, apply catalog,
run another command.
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>
>
--
Cheers,
Daniel
Michael DeHaan
> Sounds promising. What about a combination of shell execution and a
> normal puppetrun? Something like run a shell command, apply catalog,
> run another command.
>
>
use case?
OT -- one thing I didn't explain about Func is it had the notion of "do
this N at a time", in our particular case,
there may be an opportunity to teach N hosts to update at a time, and
then do N hosts later... so in very large
setups, it would be possible to do easy rolling puppetmaster updates.
Func actually handled this by forking and running synchronously, so this
would need to be a bit different.
--Michael
Daniel
"/etc/init.d/tomcat start". It also would be great to manage services
in general. Like stop/start/restart service X.
On Wed, Feb 10, 2010 at 8:49 PM, Michael DeHaan
--
Cheers,
Daniel
Joe McDonagh
> Additional ideas for stuff you would like to see?
>
> --Michael
>
>
on a wide scale. The utility becomes useless for a large portion of
people. I searched the thread quickly and didn't see this mentioned...
--
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode
L'ennui est contre-r�volutionnaire
Michael DeHaan
> Something like "/etc/init.d/tomcat stop", "deploy",
> "/etc/init.d/tomcat start". It also would be great to manage services
> in general. Like stop/start/restart service X.
>
punc --server-tags webservers --puppetrun --tags "tomcat"
And model the restarting of the service in Puppet.
punc could support running just tagged aspects of the catalog.
Michael DeHaan
> Michael DeHaan wrote:
>> Additional ideas for stuff you would like to see?
>>
>> --Michael
>>
>>
> Please take out the 'feature' that you need LDAP hosts to run
> puppetrun on a wide scale. The utility becomes useless for a large
> portion of people. I searched the thread quickly and didn't see this
> mentioned...
>
First data sources will be simple wildcards of the certs we have, second
data source will be the dashboard DB (and anything else that follows
this schema). LDAP would be a good patch for later, I'm guessing,
but probably not something I'd do initially.
--Michael
Scott Smith
> OT -- one thing I didn't explain about Func is it had the notion of "do
> this N at a time", in our particular case,
> there may be an opportunity to teach N hosts to update at a time, and
> then do N hosts later... so in very large
> setups, it would be possible to do easy rolling puppetmaster updates.
> Func actually handled this by forking and running synchronously, so this
> would need to be a bit different.
Cool, sounds like dsh.
-scott
Joe McDonagh
*requires* LDAP. That's my beef with it. Why can't it look at my
nodes.pp and get all the nodes from there?
Michael DeHaan
> Michael DeHaan wrote:
>> Joe McDonagh wrote:
>>> Michael DeHaan wrote:
>>>> Additional ideas for stuff you would like to see?
>>>>
>>>> --Michael
>>>>
>>>>
>>> Please take out the 'feature' that you need LDAP hosts to run
>>> puppetrun on a wide scale. The utility becomes useless for a large
>>> portion of people. I searched the thread quickly and didn't see this
>>> mentioned...
>>>
>> I hadn't even thought of supporting LDAP, actually :)
>>
>> First data sources will be simple wildcards of the certs we have,
>> second data source will be the dashboard DB (and anything else that
>> follows this schema). LDAP would be a good patch for later, I'm
>> guessing, but probably not something I'd do initially.
>>
>> --Michael
>>
>>
> It already does support LDAP. In fact if you try to use puppetrun -a
> it *requires* LDAP. That's my beef with it. Why can't it look at my
> nodes.pp and get all the nodes from there?
>
mantainability, I need to investigate but we definitely won't require LDAP.
As for nodes.pp --
nodes.pp can contain regexen, IIRC. I suppose technically that might
be possible, but short term, we probably won't do this.
There's nothing saying we can't look at the list of certs, which is what
Func did, but I also want it to be able to use extended metadata and
info from the dashboard interface (or anything else that uses that
schema). It may also be interesting to maintain an external json
catalog of nodes and tags, TBD... I don't want to overdesign that bit
too much up front -- "YAGNI", and what all.
(We also need a CLI for dashboard... so it's easy to add nodes and tag
them here... we don't want people using
the extended DB to have to click around a WebUI if they don't want to,
and it would be helpful with batch population).
--Michael
Joe McDonagh
with metadata, I just think the tool right now is kinda useless. If this
is improved I can ditch capistrano for scalable administration and leave
it to deploys.
Scott Smith
> (We also need a CLI for dashboard... so it's easy to add nodes and tag
> them here... we don't want people using
> the extended DB to have to click around a WebUI if they don't want to,
> and it would be helpful with batch population).
http://github.com/ohlol/puppet-dashboard/blob/master/lib/tasks/node.rake
The code sucks! I refactored it but need to push.
-scott
Michael DeHaan
Yeah that's the idea ... longer term, I think we need a user CLI that
takes formal options rather than having rake as an interface (puppet
isn't invoked with rake either, etc) ...
Trevor Vaughan
Hash: SHA1
My thoughts:
- - I would have it be a stand alone application, not an add-on to the
primary puppet daemon. The reason for this is that many of us use puppet
from cron instead of as a daemon for various reasons.
- - I do have to wonder how this is much different that cobbling together
the results of puppetlast (to get the nodes that have checked in) with
some sort of parallel SSH solution (besides the fact that you're using
the puppet certs instead of SSH keys).
- - In general, people are going to have hybrid environments for one
reason or another. The ability to add additional nodes to this list
that can perhaps just use puppetrun for this functionality may be welcome.
A little more below.
On 02/10/2010 12:45 PM, Michael DeHaan wrote:
>
> Features I'm thinking of:
>
> Requires no additonal ports, setup, or config files -- use existing
> puppet listening capability and puppetca, just a /usr/bin app
- - I don't particularly agree with this for the reasons listed above.
> Be able to query dashboard DB to run against tagged nodes or hosts
> that have certain data there (or in storeconfigs???)
- - Sounds good. Should be pluggable so that you can add additional data
sources with little effort. Flat files, LDAP, MySQL, whatever...
> Be able to run against wildcarded nodes based on what certs are
> present on the puppetmaster (we know the hostnames)
- - This makes sense.
> Be able to be used easily from an API perspective from any ruby application
- - I would vote for a combination of this and the ability to do
everything from the command line. This way people can either use Ruby or
their programming language of choice.
> Be able to invoke ralsh remotely for querying things (and for debug,
> and one off tasks)
- - Are there example use cases of this? It sounds useful, but my minimal
ralsh experimentation didn't really prove to be greatly fruitful.
> Be able to run shell commands for things that are one offs (emergency
> security power down now)
- - This comes back to the common framework. I'm going to want to be able
to do this regardless of whether or not puppet is actually running. I
may want to do this (or something like it) because puppet is hung for
some reason.
Thanks,
Trevor
- --
Trevor Vaughan
Vice President, Onyx Point, Inc.
email: tvau...@onyxpoint.com
phone: 410-541-ONYX (6699)
- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAktzOZQACgkQyWMIJmxwHpTuvgCcDO9JTYioMKt2AcaEk+sumPfj
OjYAnjTzxExPjs1isDy2n2CeU6F6UdLS
=eaQK
-----END PGP SIGNATURE-----
Michael DeHaan
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> My thoughts:
>
> - - I would have it be a stand alone application, not an add-on to the
> primary puppet daemon. The reason for this is that many of us use puppet
> from cron instead of as a daemon for various reasons.
Interesting yeah, in that case the transport then becomes SSH though?
Seems to be redundant in that configuration.
We're attempting to provide a reason to not use cron :)
>
> - - I do have to wonder how this is much different that cobbling together
> the results of puppetlast (to get the nodes that have checked in) with
> some sort of parallel SSH solution (besides the fact that you're using
> the puppet certs instead of SSH keys).
It would resemble that... but the idea here is we want to leverage
those that have already checked in
and the existing comm path. I understand your use case though... in
which case, I think just using a parallel
SSH (in that case) if you are ok with that, solves the problem... or
continuing to use cron.
Here it is really about replacing puppetrun ... :)
>
> - - In general, people are going to have hybrid environments for one
> reason or another. The ability to add additional nodes to this list
> that can perhaps just use puppetrun for this functionality may be welcome.
>
> A little more below.
>
> On 02/10/2010 12:45 PM, Michael DeHaan wrote:
>>
>> Features I'm thinking of:
>>
>> Requires no additonal ports, setup, or config files -- use existing
>> puppet listening capability and puppetca, just a /usr/bin app
>
> - - I don't particularly agree with this for the reasons listed above.
Don't follow, can you elaborate?
(I also probably missed one point here... it /might/ require some
config on puppet nodes in order
to enable the new features beyond what it does today... TBD ... we
haven't talked about this yet)
>
>> Be able to query dashboard DB to run against tagged nodes or hosts
>> that have certain data there (or in storeconfigs???)
>
> - - Sounds good. Should be pluggable so that you can add additional data
> sources with little effort. Flat files, LDAP, MySQL, whatever...
Right. We'll start with the host list and dashboard DB to start.
>
>> Be able to run against wildcarded nodes based on what certs are
>> present on the puppetmaster (we know the hostnames)
>
> - - This makes sense.
>
>> Be able to be used easily from an API perspective from any ruby application
>
> - - I would vote for a combination of this and the ability to do
> everything from the command line. This way people can either use Ruby or
> their programming language of choice.
We're on the same page :)
>
>> Be able to invoke ralsh remotely for querying things (and for debug,
>> and one off tasks)
>
> - - Are there example use cases of this? It sounds useful, but my minimal
> ralsh experimentation didn't really prove to be greatly fruitful.
I'm thinking more about querying for states of things than making changes.
>
>> Be able to run shell commands for things that are one offs (emergency
>> security power down now)
>
> - - This comes back to the common framework. I'm going to want to be able
> to do this regardless of whether or not puppet is actually running. I
> may want to do this (or something like it) because puppet is hung for
> some reason.
I understand, yeah. In this case, distributed SSH may be your friend
if you don't want to run cron and don't want puppet to listen.
Otherwise it kind of is cross-purposes with the idea of fixing
puppetrun... and extra management of SSH keys is overhead when
puppetca is there for that purpose.
I know what you're saying though...
Alan Barrett
> We're attempting to provide a reason to not use cron :)
I have a requirement that puppet may not change anything on a production
host without change control approval in advance. It would be nice if a
new version of puppet had better support for this use case.
Of course there are change control procedures for getting the manifests
updated on the puppetmaster, but that's not enough; it's also necessary
to run the puppet client only when specifically authorised. For
example, the manifest update and a --noop mode client puppet run might
happen during working hours, but the --no-noop client puppet run might
happen during a maintenance window after hours.
As far as I am aware, the existing combination of "puppetd --listen"
on each client, and "puppetrun" on a central server, does not handle
this use case, so I run "puppetd --onetime --noop" or "puppetd --onetime
--no-noop" via ssh.
--apb (Alan Barrett)
Scott Smith
So let me get this straight: You run --noop throughout the day,
aggregate the changes that need to be made, and then have a EOD/EOW
"change control" meeting to go over them and determine if you need to
run puppet without --noop ?
-scott
Ohad Levy
May I recommend that you have a look at the ext directory for puppetlisten/puppetrun[1], this two scripts I wrote a while ago reuse puppet certificate infrastructure to trigger remote runs.
additionally, I've created a query interface in foreman[2], which could probably give you some ideas of how integration can be done.
by combining this two solutions, I think that you will have a very powerful solution.
one thing to keep in mind is that many puppet users are using it via cron, because of the high memory footprint.
thanks,
Ohad
*1 http://github.com/ohadlevy/puppet/tree/puppetlisten/ext/puppetlisten/
*2 http://theforeman.org/wiki/foreman/Query_Interface
Alan Barrett
> >Of course there are change control procedures for getting the manifests
> >updated on the puppetmaster, but that's not enough; it's also necessary
> >to run the puppet client only when specifically authorised. For
> >example, the manifest update and a --noop mode client puppet run might
> >happen during working hours, but the --no-noop client puppet run might
> >happen during a maintenance window after hours.
>
> aggregate the changes that need to be made, and then have a EOD/EOW
> "change control" meeting to go over them and determine if you need
> to run puppet without --noop ?
Not exactly, but that would probably be acceptable. What actually
happens today is that puppetd is not run at all on the client unless an
authorised change is known to be ready for deployment; then puppetd is
run in --noop mode to verify that the changes it wants to make are as
expected; finallly puppetd is run in --no-noop mode.
--apb (Alan Barrett)
Scott Smith
> Not exactly, but that would probably be acceptable. What actually
> happens today is that puppetd is not run at all on the client unless an
> authorised change is known to be ready for deployment; then puppetd is
> run in --noop mode to verify that the changes it wants to make are as
> expected; finallly puppetd is run in --no-noop mode.
>
What happens if a manual change is made outside of the scope of your scheduled Puppet changes? For
example, someone tweaks /etc/syslog.conf. Puppet manages it but because you don't run regular noops,
how will you know it needs to be changed back? Do plan around the organization (Ops or IT or w/e)
not knowing it was changed? Since you don't have regular reports, how do you find out?
-scott
Trevor Vaughan
Hash: SHA1
Inline....
>> - - I would have it be a stand alone application, not an add-on to the
>> primary puppet daemon. The reason for this is that many of us use puppet
>> from cron instead of as a daemon for various reasons.
>
> Interesting yeah, in that case the transport then becomes SSH though?
> Seems to be redundant in that configuration.
- -THV- Agreed, but SSH seems to be a bit more reliable than puppet as a
transport layer due to user error. I've had cases where I implemented
something that worked fine on 20 nodes but then, on node 21, hung for
some reason and could not recover. Puppetrun, in this case, would be
completely useless.
This is a simplified version of reality, but it happens as not all
variables can be successfully tested, or known, in any given environment.
>
> We're attempting to provide a reason to not use cron :)
- -THV- Why? Puppet currently potentially uses quite a bit of memory on a
large manifest set and seems to work great out of cron with no adverse
side effects.
>
>>
>> - - I do have to wonder how this is much different that cobbling together
>> the results of puppetlast (to get the nodes that have checked in) with
>> some sort of parallel SSH solution (besides the fact that you're using
>> the puppet certs instead of SSH keys).
>
> It would resemble that... but the idea here is we want to leverage
> those that have already checked in
> and the existing comm path. I understand your use case though... in
> which case, I think just using a parallel
> SSH (in that case) if you are ok with that, solves the problem... or
> continuing to use cron.
>
> Here it is really about replacing puppetrun ... :)
- -THV- I definitely understand this, but I suppose that I'm questioning
the existence of puppetrun altogether. It's relatively useful, but I'm
guessing that one of the main things that most people do with puppet is
to set up SSH. As such, you're re-inventing the wheel with a
questionable gain for the effort.
>>>
>>> Requires no additonal ports, setup, or config files -- use existing
>>> puppet listening capability and puppetca, just a /usr/bin app
>>
>> - - I don't particularly agree with this for the reasons listed above.
>
> Don't follow, can you elaborate?
- -THV- No additional ports means an additional listening thread inside of
puppet itself. If we were using Ruby 1.9 with native threads, then this
wouldn't be a problem but the way it is currently implemented I don't
have much faith that a hang in one part of Puppet wouldn't cause the
listener to hang as well. I could be wrong :-P.
The reduction of setup and config (as it is now) makes sense.
>>
>> - - Are there example use cases of this? It sounds useful, but my minimal
>> ralsh experimentation didn't really prove to be greatly fruitful.
>
> I'm thinking more about querying for states of things than making changes.
- -THV- Ah. This makes sense. I have had some problems with ralsh in the
past not picking up my complete manifest. I'll try to come up with some
concrete examples if possible.
>
>>
>>> Be able to run shell commands for things that are one offs (emergency
>>> security power down now)
>>
>> - - This comes back to the common framework. I'm going to want to be able
>> to do this regardless of whether or not puppet is actually running. I
>> may want to do this (or something like it) because puppet is hung for
>> some reason.
>
> I understand, yeah. In this case, distributed SSH may be your friend
> if you don't want to run cron and don't want puppet to listen.
> Otherwise it kind of is cross-purposes with the idea of fixing
> puppetrun... and extra management of SSH keys is overhead when
> puppetca is there for that purpose.
>
> I know what you're saying though...
- -THV- I think that the point I'm so blatantly failing to make is that
puppet isn't a tool for one-offs and isn't advertised as a command and
control infrastructure. I suppose that my *NIX roots are showing in that
I shy away from adding features to a tool that I can easily do with
something else.
As I mentioned in this response, one of the first things people are
probably going to do is to get a working SSH infrastructure going. Once
this is done, you have everything you need to do one-offs.
Thinking of some common use cases:
1) I screwed up resolv.conf
- Puppet's not going to work unless you pre-mangled /etc/hosts, SSH it is.
2) I screwed up /etc/hosts
- Puppet's not going to work. Back to SSH.
3) DNS is broken
- See #1
4) I hosed my PKI certs
- See #2
In these four cases, you *have* to have a working SSH infrastructure
that spans your entire puppet base.
So, after all of this rambling, perhaps you may want to have the new
tool be transport pluggable. Support both SSH and Puppet in the present
and perhaps grow to support things like mcollective under the same
abstracted administrative interface.
Thanks,
Trevor
- --
Trevor Vaughan
Vice President, Onyx Point, Inc.
email: tvau...@onyxpoint.com
phone: 410-541-ONYX (6699)
- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkt2cOMACgkQyWMIJmxwHpSKhACgh6QXNkaQMyh7+tdL33QTIqZm
66AAn0g379T1RQxA0vrJLGXtdoG8AWhj
=tAPZ
-----END PGP SIGNATURE-----
Michael DeHaan
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Inline....
>
>>> - - I would have it be a stand alone application, not an add-on to the
>>> primary puppet daemon. The reason for this is that many of us use puppet
>>> from cron instead of as a daemon for various reasons.
>>
>> Interesting yeah, in that case the transport then becomes SSH though?
>> Seems to be redundant in that configuration.
>
> - -THV- Agreed, but SSH seems to be a bit more reliable than puppet as a
> transport layer due to user error. I've had cases where I implemented
> something that worked fine on 20 nodes but then, on node 21, hung for
> some reason and could not recover. Puppetrun, in this case, would be
> completely useless.
>
> This is a simplified version of reality, but it happens as not all
> variables can be successfully tested, or known, in any given environment.
>
>>
>> We're attempting to provide a reason to not use cron :)
>
> - -THV- Why? Puppet currently potentially uses quite a bit of memory on a
> large manifest set and seems to work great out of cron with no adverse
> side effects.
We need to work on that. I don't want to constrain our conceptual
ideas too much
by current limitations.
Anyway, if you are able to stagger node rollouts according to custom
workflow you gain two
capabilities:
one -- you can not role out your infrastructure all at once, for
safety reasons (even if you
do have a good 'stage' infrastructure). For instance, update just
some of these hosts,
and then update the rest.
two -- you have a nicer scaling puppetmaster as you have more control
on when they check
in and put load on the server, rather than having the spike happen at
intervals (and without
you having to manage these intervals with cron). While we're
working to speed up puppetmaster
and also make it scale out easier, just being able to stage rollouts
centrally combined with that
can offer some nice performance opportunity.
Finally three (amongst our capabilities </spanish inquisition
sketch>), you don't have to manage
your crontab for puppet in your image/kickstart/etc, because it can
all be centrally managed and not
up to the client.
Is that going to work for all use cases? Probably not, but if you
have something you like, you can definitely
continue to use it. I'd like to have better control over when things
run from a central server perspective myself,
so I could do things faster at times, have more fined grained control, etc.
>
> - -THV- I definitely understand this, but I suppose that I'm questioning
> the existence of puppetrun altogether. It's relatively useful, but I'm
> guessing that one of the main things that most people do with puppet is
> to set up SSH. As such, you're re-inventing the wheel with a
> questionable gain for the effort.
Perhaps... I agree with your pluggable transport things below.
It also seems that (at least from what I gather) that not everyone
/does/ distribute
SSH keys. (They might rely on serial consoles, for instance).
>
> - -THV- No additional ports means an additional listening thread inside of
> puppet itself. If we were using Ruby 1.9 with native threads, then this
> wouldn't be a problem but the way it is currently implemented I don't
> have much faith that a hang in one part of Puppet wouldn't cause the
> listener to hang as well. I could be wrong :-P.
The real idea is not have hangs in that part of Puppet, isn't it? :)
I'll keep that in mind as I'm doing it. I'm sure there will be some
corner cases here and there that we'll have
to straighten out.
>
>>>
>>> - - This comes back to the common framework. I'm going to want to be able
>>> to do this regardless of whether or not puppet is actually running. I
>>> may want to do this (or something like it) because puppet is hung for
>>> some reason.
*nod*
That would be just one of many small features. If you wanted to use
SSH or serial consoles for that, you still could, of course.
For those that don't have that, having a distributed system out of the
box that can do that is pretty darn useful.... and we can
get that more or less for free.
Best practices, as you say, should probably imply having a backup plan.
> So, after all of this rambling, perhaps you may want to have the new
> tool be transport pluggable. Support both SSH and Puppet in the present
> and perhaps grow to support things like mcollective under the same
> abstracted administrative interface.
Agreed. Might not get that in initially, but we should keep it in
mind and do the very first one as a plugin, to make room
for the rest.
Dave Thomas
my MS thesis. Last weekend I met Dan at SCALE, who suggested I look
up this thread. Going to your earlier example:
> punc --hosts *.example.org --shell "/bin/emergency_script"
How about extending this to use puppet's RAL or yaml:
punc --yamlfile maintenance.yaml --process emergency_script
Where yamlfile could resemble:
<snip>
processes:
emergency_script:
execute:
- /bin/sh /bin/emergency_script
locations:
hosts:
- singlehost.example.org
</snip>
Using YAML anchors could be useful for specifying workflows:
<snip>
processes:
install_and_sign:
execute:
- *install
- *sign
- *start
install: &install
execute:
- yum install puppet
- puppetd --waitforcert 60
locations:
hosts:
- $newhost
sign: &sign
execute:
- puppetca --sign $newhost
locations:
hosts:
- puppetmaster
start: &start
execute:
- /etc/init.d/puppet start
locations:
hosts:
- $newhost
</snip>
This example uses variables like $newhost which can be improved upon,
but for now you could support it from the command-line like:
punc --yamlfile sign.yaml --process install_and_sign --set
newhost=newhost.example.com
Setting variables or separate yaml can separate a process from where
it runs (staging vs. production). But the main idea is to support
classes with more specific behavior, like so:
<snip>
locations:
--- !ruby/object:YAML::puppetclasses
name: example.org
--- !ruby/object:YAML::ldap
name: example
--- !ruby/object:YAML::dsh
name: example
</snip>
To produce non-ruby-specific YAML can simplify the last snip that
to ...
<snip>
locations:
--- !puppetclasses
name: example.org
--- !ldap
name: example
--- !dsh
name: example
</snip>
... though I don't know if there's a way (http://www.yaml.org/
YAML_for_ruby.html#objects) to bind YAML <==> Ruby objects, except by
hand, like you can with other implementations:
http://code.google.com/p/snakeyaml/wiki/Documentation#Constructors,_representers,_resolvers
http://pyyaml.org/wiki/PyYAMLDocumentation#Constructorsrepresentersresolvers
In a world of competing deployment tools, being language-agnostic
could unify deployment processes. Java people resist writing python,
etc so there are so many competing tools. I'm trying to distill this
to the simplest thing that can possibly be useful, so:
- I like YAML because it's human-readable and all languages support
it. I like Puppet's RAL, too.
- As described, it was simple enough to be implemented in about 30
lines in python implementation with fabric (fabfile.org), so hopefully
that remains so with puppetrun/vlad/capistrano/ace/func.
- I've left out ways you might express a transport (ssh vs puppet vs
web services)
- Brevity limits how to incorporate concepts inspired by BPEL like
decisions (to support if/then), fork/join (execute tasks in parallel),
tasks (prompt user input), and state (variables).
I hope these ideas contribute to a useful orchestration model for
puppet. I want as much feedback as I can get, my direct email is dave
(AT) peoplemerge (DOT) com.
Dave Thomas