Some noob puppet questions
Bill
1) I'm planning on using puppet on 1,000's of machines. If, say, 500
of them are to be configured the same, do I really have to manage 500
node entries on a single node line in site.pp or is there a cleaner
way? Since our own home-grown configuration management software
stores system attributes in a database, it seems much nicer.
2) Is there a GUI front-end to manage this? Again, it seems like
manually managing a file with 1,000's of node names in it could prove
dangerous.
3) We currently have a home-grown file distribution mechanism in place
to control our hosts, passwd, aliases, etc. files. We do this by
creating the file in a repository, in a directory named for the target
file, with the file name a list of attributes. So, if you wanted to
manage the passwd file on system "foo" you'd have a file '<repository>/
etc/passwd/hostname=foo'. If you wanted to manage all systems named
"bar" with usage "hacluster", you'd have a file name
'hostname=bar,usage=hacluster'. (We store a system's "usage"
attribute
This kind of setup looks difficult to manage in puppet. For example,
if you had something like this:
node default {
baseserver
}
class baseserver {
include someclass
include standardpassword
include anotherclass
include yetanotherclass
}
You'd have to setup a completely new node just to manage the password,
with lots of repetition:
node foo {
include someclass
include modifiedpassword
include anotherclass
include yetanotherclass
}
This doesn't even touch assigning "attributes" to a system and being
able to distribute files that way.
Is anyone doing something similar? Am I not seeing something?
Paul Lathrop
>
> Hello all. Day-old user of puppet here with some questions.
Hello, and welcome.
> 1) I'm planning on using puppet on 1,000's of machines. If, say, 500
> of them are to be configured the same, do I really have to manage 500
> node entries on a single node line in site.pp or is there a cleaner
> way? Since our own home-grown configuration management software
> stores system attributes in a database, it seems much nicer.
You probably want to read up on External Nodes. See
http://reductivelabs.com/trac/puppet/wiki/ExternalNodes for
information.
> 2) Is there a GUI front-end to manage this? Again, it seems like
> manually managing a file with 1,000's of node names in it could prove
> dangerous.
There is no GUI at the moment. However, as I understand it, Reductive
Labs is creating a commercial GUI tool that goes alongside Puppet; you
may want to contact them for details.
> 3) We currently have a home-grown file distribution mechanism in place
> to control our hosts, passwd, aliases, etc. files. We do this by
> creating the file in a repository, in a directory named for the target
> file, with the file name a list of attributes. So, if you wanted to
> manage the passwd file on system "foo" you'd have a file '<repository>/
> etc/passwd/hostname=foo'. If you wanted to manage all systems named
> "bar" with usage "hacluster", you'd have a file name
> 'hostname=bar,usage=hacluster'. (We store a system's "usage"
> attribute
That's certainly interesting. I'm sure you'd end up wanting to replace
this by Puppet's file distribution. Modules keep concerns nicely
separated.
> This kind of setup looks difficult to manage in puppet. For example,
> if you had something like this:
>
> node default {
> baseserver
> }
This would be 'include baseserver'
> class baseserver {
> include someclass
> include standardpassword
> include anotherclass
> include yetanotherclass
> }
>
> You'd have to setup a completely new node just to manage the password,
> with lots of repetition:
>
> node foo {
> include someclass
> include modifiedpassword
> include anotherclass
> include yetanotherclass
> }
Actually, it would read like this:
node default {
include baseserver
}
class baseserver {
include someclass
include anotherclass
include yetanotherclass
}
node bar inherits default {
include standardpassword
}
node foo inherits default {
include modifiedpassword
}
> This doesn't even touch assigning "attributes" to a system and being
> able to distribute files that way.
There is a wide variety of choices for how to assign attributes. You
could use custom facts, external nodes, or even functions.
> Is anyone doing something similar? Am I not seeing something?
You definitely should spend some more time browsing the wiki. I think
Puppet is more than up to the tasks you have described. Your biggest
hurdles are going to be making conceptual shifts to start thinking of
things the Puppet Way.
Regards,
Paul
Bill
wrapping my head around the Puppet "concept"!
James Turnbull
Hash: SHA1
Bill wrote:
> Thanks for the tips, Paul. I definitely need to spend more time
> wrapping my head around the Puppet "concept"!
>
Bill
Whilst it's not a definitive guide - just an approach - the Puppet Best
Practise document is a good starting point:
http://reductivelabs.com/trac/puppet/wiki/PuppetBestPractice
Regards
James Turnbull
- --
Author of:
* Pulling Strings with Puppet
(http://www.amazon.com/gp/product/1590599780/)
* Pro Nagios 2.0
(http://www.amazon.com/gp/product/1590596099/)
* Hardening Linux
(http://www.amazon.com/gp/product/1590594444/)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFIWmS89hTGvAxC30ARAvEuAKC2qdp+VPT9txrk1Qw1ns68POWDHwCgibpj
N/5mM9bjFV/JyYdv1n9tmXc=
=gYyT
-----END PGP SIGNATURE-----
Bill
Another part of our home-grown management system is to run scripts on
systems that check the health of the system. For example, a script
might run once an hour, check for file systems that are 90% full. If
one is found, we send out an "alert" to a central server which posts
the alert to a web page. Or, if a system doesn't have the right
patches/patch levels, instead of automatically updating them, we might
just want a report of all systems that are downrev.
The only thing I found so far that sounded even close was using
'functions' but they only seem to run on the server side...
Anyway, do you see a use for puppet for this? If not, I worry about
having to integrate a puppet solution with a non-puppet one,
especially if a systems "attributes" have to be accessible by both.
Evan Hisey
>
> Thanks for the tips, Paul. I definitely need to spend more time
> wrapping my head around the Puppet "concept"!
>
modles from day one, and second stop thinking about services,
applications or user by the individual components. These 2 points
actually go hand in hand. At the node description level you want to
focus on what service each machine provides, ie: ssh, httpd, email.
You then take these service level resource and create modules so that
you would have a module for ssh, one for email, and one for http. In
side each module break out what resources are needed for each service
in to classes. Only after you have reached the point of describing the
in class resources should you begin to worry about the actual system
level stuff.
Thsi was the biggest change i had to make going from a home-grown to
puppet. After you get used to thinking this way, puppet is a snap.
Also Jame's book Pulling strings with Puppet is a really good way to
see this approach in cation and get a feel for the puppet concepts.
Evan
Evan Hisey
>
> A follow-up, more along conceptual lines.
>
> Another part of our home-grown management system is to run scripts on
> systems that check the health of the system. For example, a script
> might run once an hour, check for file systems that are 90% full. If
> one is found, we send out an "alert" to a central server which posts
> the alert to a web page. Or, if a system doesn't have the right
> patches/patch levels, instead of automatically updating them, we might
> just want a report of all systems that are downrev.
>
> The only thing I found so far that sounded even close was using
> 'functions' but they only seem to run on the server side...
>
> Anyway, do you see a use for puppet for this? If not, I worry about
> having to integrate a puppet solution with a non-puppet one,
> especially if a systems "attributes" have to be accessible by both.
>
seperate monitoring system. A tool that does one job well, is much
better than a tool handling 2 jobs poorly. Using an external node
database like LDAP solves the problem of sharing attributes. Puppet
has very good hooks for managing a nagios monitor. I use hobbit fo r
this task myself.
Once you have puppet up and running you should not have problems with
miss matched patch levels. Maintaining everything at certain level or
the same level across all platforms is what puppet was built to do.
You couls use the puppet reporting system to a report of what puppet
is doing/seeing on every node.
If you have custom scripts you want to run on each client to report
back to some where. you can use puppet to send the scripts out and use
the exec{} resource to run those scripts.
Evan
Duncan Hill
>
> A follow-up, more along conceptual lines.
>
> Another part of our home-grown management system is to run scripts on
> systems that check the health of the system. For example, a script
> might run once an hour, check for file systems that are 90% full. If
> one is found, we send out an "alert" to a central server which posts
> the alert to a web page. Or, if a system doesn't have the right
> patches/patch levels, instead of automatically updating them, we might
> just want a report of all systems that are downrev.
We use puppet to deploy the HP tools to our machines, along with
configuration. OpenNMS deals with the polling and traps, and Insight
Manager (old install, getting replaced) handles things like setting
disk thresholds. As others have indicated, puppet is a good
configuration management tool, it's not a monitoring tool - though I'm
sure you could use it to report custom facts back about package
versions into a storeconfigs database and then query that to see when
you're behind on revisions. Alternately, I know that CentOS/RedHat
snmpd report all installed packages and versions, so you could do that
particular report via snmp.
Bill
Is Nagios the most widely used open source monitoring system? Since
all of our monitoring is home-grown, we haven't had the need to "get
out" much ;)
Evan Hisey
>
> Gotcha, so look elsewhere for monitoring.
>
> Is Nagios the most widely used open source monitoring system? Since
> all of our monitoring is home-grown, we haven't had the need to "get
> out" much ;)
>
The three real well known ones seem to be Nagios,
Bigbrother/BigSister/Hobbit (all share code), and Zenoss. Nagios has
the best support by puppet. I use Hobbit. It is also pretty easy to
manage via puppet. Zenoss I was never able to really get a grip on.
Evan
Jeff Adams
uses SNMP to monitor hosts, so we only need puppet to ensure that SNMP
is installed and configured on all our hosts. We then use Zenoss itself
to manage what it's monitoring, primarily via it's auto-discovery and
modeling features, which works nicely for us.
Admittedly, Zenoss works/organizes things differently than the other
monitoring solutions that I'm familiar with. But, much like puppet, once
you get your head wrapped around how it wants to work, it works quite well.
- Jeff
Paul Lathrop
recommend sticking with Nagios. Zenoss seemed to consistently fail to
run checks, reported problems when I could verify they were not
happening, and failed to report problems even when I intentionally
created them.
This was in January, so it is possible they worked out some of these
issues since.
--Paul
James Turnbull
Hash: SHA1
Bill wrote:
> Gotcha, so look elsewhere for monitoring.
>
> Is Nagios the most widely used open source monitoring system? Since
> all of our monitoring is home-grown, we haven't had the need to "get
> out" much ;)
I have a preference for Nagios but I'm biased. I've also used a bit of
Groundwork Open Source which bundles a variety of tools and a pretty
front-end - http://www.groundworkopensource.com/ - if that's a
requirement for you.
The new 3.0 version of Nagios is soon to be gold and probably well worth
a look.
Regards
James
- --
Author of:
* Pulling Strings with Puppet
(http://www.amazon.com/gp/product/1590599780/)
* Pro Nagios 2.0
(http://www.amazon.com/gp/product/1590596099/)
* Hardening Linux
(http://www.amazon.com/gp/product/1590594444/)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFIWvEr9hTGvAxC30ARAssfAKDTxxQFNzTLjvxhb9kmFJM7uYbiPACgmqJa
MMk6rLvwB9K1PqyohxymRwA=
=Rqwn
-----END PGP SIGNATURE-----
Matt Ray
2.2 both came out since then. Zenoss is ramping up on a lot of
fronts, QA and documentation especially so it'll keep getting better.
As far as Puppet goes, I've been investigating putting together a
ZenPack (a Zenoss extension) to have them work together. I'd been
thinking something along the lines of having Puppet and Zenoss share
lists of machines and look for ways to keep them in sync, so when a
new machine is added to Puppet, it gets automatically pushed into
Zenoss for monitoring, and when Zenoss discovers a new device, it
might send an alert to the Puppet administrator to see if the machine
is accounted for. I don't know if the demand is there, but if
anyone's interested in working on it or has more ideas, I'd be glad to
help facilitate that.
Thanks,
Matt Ray
Zenoss Community Manager
mr...@zenoss.com
Ashley Penney
Evan Hisey
> wrapping my head around the Puppet "concept"!
>
to go back and follwo myself. when buildin gout servers or service to
use with puppet, don't hadn craft them then try and make puppet
maintain them. This route can lead to all kinds of headaches and makes
it difficult to see things the puppet way. Instead take a barebone
installed box and then create the service via puppet. I know this
sounds backwards but it lets you focus on the service through the
puppet concepts and also guarentees that you don't forget something
when creating the puppet manifests for the service. An added bonus of
doing it this way is that once the service is ready your done and
puppet already takes care of the service.
Evan