Puppet, inventory, and single sources of truth.

[Robin Lee Powell]

So I've been using puppet for a long time, and the one thing I've
never solved to my satisfaction is a way to have a single source of
truth that acts as both instructions to puppet *and* as a system
inventory that I can use for general opertaions (i.e. "how many
tomcat hosts do we have?").

When Hiera came along I shifted to that, believing that it was the
right solution here, and I've managed to hack together something
that works, but it's pretty inelegant. The reason is that to get a
proper inventory out of hiera requires collating all the hiera data
from the point of view of each host, so that all the hierarchical
processing is correct, and then mushing all those results together.
I've got a system to do that, but it's pretty hacky.

Is there some better way of combining a general inventory system and
puppet? Is this a Puppet Enterprise sort of thing?

--
http://intelligence.org/ : Our last, best hope for a fantastic future.
.i ko na cpedu lo nu stidi vau loi jbopre .i dafsku lu na go'i li'u .e
lu go'i li'u .i ji'a go'i lu na'e go'i li'u .e lu go'i na'i li'u .e
lu no'e go'i li'u .e lu to'e go'i li'u .e lu lo mamta be do cu sofybakni li'u

[Garrett Honeycutt]

On 10/26/14 4:46 PM, Robin Lee Powell wrote:
>
> So I've been using puppet for a long time, and the one thing I've
> never solved to my satisfaction is a way to have a single source of
> truth that acts as both instructions to puppet *and* as a system
> inventory that I can use for general opertaions (i.e. "how many
> tomcat hosts do we have?").
>
> When Hiera came along I shifted to that, believing that it was the
> right solution here, and I've managed to hack together something
> that works, but it's pretty inelegant. The reason is that to get a
> proper inventory out of hiera requires collating all the hiera data
> from the point of view of each host, so that all the hierarchical
> processing is correct, and then mushing all those results together.
> I've got a system to do that, but it's pretty hacky.
>
> Is there some better way of combining a general inventory system and
> puppet? Is this a Puppet Enterprise sort of thing?
>

Hi,

Are you using PuppetDB? You can query it to see how many systems have
the tomcat class associated with them. It also stores facts, so if you
classify your node by setting a custom fact, such as `role`, you could
query PuppetDB for all systems where role => 'app_server' or whatever.

If you want real time data about the nodes instead of from the last
puppet check in, MCollective can help you by querying your systems in
real time.

Best regards,
-g

--
Garrett Honeycutt
@learnpuppet
Puppet Training with LearnPuppet.com
Mobile: +1.206.414.8658

[Brian Morris]

I accomplish this through a way that may fit your needs. I use facters for it. Here is one for looking for Apache on Debian-based distro:

 

Facter.add(:apache_exists) do

confine :osfamily => "Debian"

setcode do

if Facter::Util::Resolution.exec("dpkg -l | grep apache2 | grep -v apache2-utils | grep ^ii")

"true"

end

end

end

This shows up in PuppetDB and Puppet Dashboard as "apache_exists", and "true" for systems that have it.

Here is one for looking for the version of SEP on a Windows server:

 

Facter.add("app_sep_version") do

  confine :osfamily => "Windows"

setcode do

Facter::Util::Resolution.exec('C:\Windows\sysnative\WindowsPowerShell\v1.0\powershell.exe -Command "& {Get-ChildItem hklm:\software\microsoft\windows\currentversion\uninstall | ForEach-Object {Get-ItemProperty $_.pspath} | Where-Object {$_.DisplayName -eq \"Symantec Endpoint Protection\"} | ForEach-Object -process {$_.DisplayVersion } }" ' )

end

end

That one reports the version of SEP to both Puppet DB, and Puppet Dashboard.

I hope this helps,

Brian 

[Robin Lee Powell]

On Sun, Oct 26, 2014 at 06:09:08PM -0700, Garrett Honeycutt wrote:
> On 10/26/14 4:46 PM, Robin Lee Powell wrote:
> >
> > So I've been using puppet for a long time, and the one thing
> > I've never solved to my satisfaction is a way to have a single
> > source of truth that acts as both instructions to puppet *and*
> > as a system inventory that I can use for general opertaions
> > (i.e. "how many tomcat hosts do we have?").
> >
> > When Hiera came along I shifted to that, believing that it was
> > the right solution here, and I've managed to hack together
> > something that works, but it's pretty inelegant. The reason is
> > that to get a proper inventory out of hiera requires collating
> > all the hiera data from the point of view of each host, so that
> > all the hierarchical processing is correct, and then mushing all
> > those results together. I've got a system to do that, but it's
> > pretty hacky.
> >
> > Is there some better way of combining a general inventory system
> > and puppet? Is this a Puppet Enterprise sort of thing?
> >
>
> Hi,
>
> Are you using PuppetDB?

[or mcollective]

I want to store data about what's *supposed* to be true about our
systems, not what is *actually* true. i.e. "host X is supposed to
be up and in subnet Y", even if it's never actually been turned on.

[Robin Lee Powell]

(Copying my response to the other branch of this thread).

I want to store data about what's *supposed* to be true about our
systems, not what is *actually* true. i.e. "host X is supposed to
be up and in subnet Y", even if it's never actually been turned on.

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/3b7f0966-df19-49e1-b50b-c1b20304f4f0%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Gavin Williams]

Sounds like The Foreman [1] might be a good option if you want to manage 'should', not 'is'... 

[1] http://theforeman.org/

[Robin Lee Powell]

On Wed, Oct 29, 2014 at 02:09:25AM -0700, Gavin Williams wrote:
> Sounds like The Foreman [1] might be a good option if you want to manage
> 'should', not 'is'...
>
> [1] http://theforeman.org/

*nod* Thanks, I should look at that again; it's been a while.

Unfortunately our internal abstraction isn't host/VM/node based at
all; it's based on sites (i.e. foo.company.com is a site, as is
bar.company.com), which have one or more associated nodes.

Having read
https://docs.puppetlabs.com/puppet/latest/reference/subsystem_catalog_compilation.html
and poking around a bit, I've had the following additional thoughts:

1. External Facts (
https://docs.puppetlabs.com/facter/latest/custom_facts.html#external-facts
)

2. generate() + stdlib's loadyaml() to just grab the values out of
a bunch of on-disk files via a shell script or something

3. Mush all the values into one tree *in Hiera*. Currently all the
values are in hiera but split up, so one node might see:

sites:
alice:
premium: true

And another might see:

sites:
bob:
premium: false

But certain parts of the code need to see:

sites:
alice:
premium: true
bob:
premium: false

I could simply de-hierachicalize (lol) all such information so that
it's in one big blob in hiera somewhere.

That gets ugly, though, just because it's one giant file; it's a lot
easier on my users when each site is its own file.

I wonder if there's a way to tell hiera "Please load every file in
this directory for every node"?

[Ohad Levy]

On Thu, Oct 30, 2014 at 2:28 AM, Robin Lee Powell <rlpo...@digitalkingdom.org> wrote:

On Wed, Oct 29, 2014 at 02:09:25AM -0700, Gavin Williams wrote:
> Sounds like The Foreman [1] might be a good option if you want to manage
> 'should', not 'is'...
>
> [1] http://theforeman.org/

*nod*  Thanks, I should look at that again; it's been a while.

Unfortunately our internal abstraction isn't host/VM/node based at
all; it's based on sites (i.e. foo.company.com is a site, as is
bar.company.com), which have one or more associated nodes.

Foreman supports multiple Organizations and locations out of the box, especially since this is a common usage case.

Ohad  

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20141030002856.GC6484%40stodi.digitalkingdom.org.