Automatically cleaning up if a class/define is no longer used (related to purging unmanaged resources)

[Vladimir Brik]

Hello,

I have a question about automatically cleaning configuration if the
class/define that generated it is no longer used.

For example, I have a number of logging daemons that I would like to be
able to disable automatically if their definitions are removed from the
node's manifest:

node foo {
# "bar" and "baz" are actually names of pre-define logging
# configurations statlog knows what to do with
statlog { "bar":}
statlog { "baz":}
}

This, basically, creates, starts, and enables (on boot) two new services
"statlog.bar" and "statlog.baz". So far so good.

Now, because statlog can be resource-intensive, suppose I would like the
statlog.baz service to be stopped and disabled automatically if I remove
statlog { "baz":} from the manifest:

site.pp:
node foo {
statlog { "bar":}
}

Since there are many possible "statlogs", I don't want to make site.pp
messy by explicitly doing something like the following for every one.
site.pp:
node foo {
statlog { "bar":}
statlog { "baz": purge => true}
}

Here's roughly what I am thinking of doing (but I am wondering if there
is a better way).

site.pp:
# this is in global area of site.pp. so that all nodes, whether they
# use statlog or not include this code
class { "statlog::autocleanup":}

node foo {
statlog { "bar":}
}

statlog module:
class statlog::autocleanup()
{
# real implementation is a bit more complicated to handle
# resource ordering, non-existent services, etc.
if !defined(Service["statlog.baz"]) {
service{ "statlog.baz":
ensure => stopped,
enable => false,
}
}
if !defined(Service["statlog.foo"]) {
...
}
}

Is there a better way to do this?



Thanks very much,

Vlad

[Christopher Wood]

On Mon, Sep 22, 2014 at 02:04:57PM -0500, Vladimir Brik wrote:
> Hello,
>
> I have a question about automatically cleaning configuration if the
> class/define that generated it is no longer used.
>
> For example, I have a number of logging daemons that I would like to
> be able to disable automatically if their definitions are removed
> from the node's manifest:
>
> node foo {
> # "bar" and "baz" are actually names of pre-define logging
> # configurations statlog knows what to do with
> statlog { "bar":}
> statlog { "baz":}
> }

You'll have better results by building in some kind of enable=>false parameterization. Removing something from the catalog means that puppet no longer manages it, but specifying that a service is stopped will actually implement some result.

define statlog ($enabled) {
service { "statlog_${name}":
ensure => $enabled,
enable => $enabled,

}
}

> This, basically, creates, starts, and enables (on boot) two new
> services "statlog.bar" and "statlog.baz". So far so good.
>
> Now, because statlog can be resource-intensive, suppose I would like
> the statlog.baz service to be stopped and disabled automatically if
> I remove statlog { "baz":} from the manifest:
>
> site.pp:
> node foo {
> statlog { "bar":}
> }

It's barely resource-intensive to stop something. After it has been stopped for a while you can of course remove it from the manifest with no effect.

I dealt with this by listing things in hiera when I have many similar things. Going on the earlier example, some yaml which you can automatically generate if you have some automated way of listing which statlogs should be enabled and which should not:

---
statlogs:
foo:
bar:
enabled: false
baz:

Then a class:

class manystatlogs {
$statlogs = hiera('statlogs')
$defaults = { enabled => true, }
create_resources('statlog', $statlogs, $defaults)
}

(These days I think the hiera/create_resources functions are easier to use, iffy on the params classes, worried about many lookups with class parameters, topic still unsolved.)

Then in your profile or node or whatever:

include ::manystatlogs

This lets you edit the hiera and leave the code alone. Obviously you're still actively managing things which are not to run, which is how puppet works.

> Thanks very much,
>
> Vlad
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/542072D9.6050500%40icecube.wisc.edu.
> For more options, visit https://groups.google.com/d/optout.

[José Luis Ledesma]

Hi,

  what I do in this case is create a fact that returns which statlog  are configured. then I do the difference (stdlib) between this and what should the machine have. Would be something like this (caution, not tested :)):

$shouldhave=['bar','foo']

$currentlyhave=$::some_fact_with_an_array_of_what_the_machine_have

$shouldnothave=difference($currentlyhave,$shouldhave)

statlog {$shouldhave:}

statlog {$shouldnothave:

       purge => true #o whatever

}

regards,

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/542072D9.6050500%40icecube.wisc.edu.
For more options, visit https://groups.google.com/d/optout.

--
José Luis Ledesma

[jcbollinger]

On Monday, September 22, 2014 2:05:42 PM UTC-5, Vladimir Brik wrote:

 

Here's roughly what I am thinking of doing (but I am wondering if there
is a better way).

site.pp:
        # this is in global area of site.pp. so that all nodes, whether they
        # use statlog or not include this code
        class { "statlog::autocleanup":}

        node foo {
                statlog { "bar":}
        }

statlog module:
class statlog::autocleanup()
{
        # real implementation is a bit more complicated to handle
         # resource ordering, non-existent services, etc.
        if !defined(Service["statlog.baz"]) {
                service{ "statlog.baz":
                     ensure => stopped,
                    enable => false,
                 }
        }
        if !defined(Service["statlog.foo"]) {
        ...
     }
}

Is there a better way to do this?

You have used the defined() function.  There is always a better way than that.

It is unclear to me why, if you want to purge undeclared Statlog resources, what you actually do is manage Service resources.  If you want to be able to remove Statlogs, then that type should have an 'ensure' parameter that you can set to 'absent' to cause that resource to be removed.  Supposing you have that, a better way to proceed would be more like this:

class statlogs::none($all_statlogs) {
  statlogs::statlog { $all_statlogs: ensure => 'absent' }
}

node foo {
  include 'statlogs::none'

  Statlogs::Statlog <| title == 'bar' |> { ensure => 'present' }
}

If you wished, it might be a bit nicer to wrap the resource override in its own defined type, maybe something like statlogs::enabled.  As I wrote it, class parameter $statlogs::none::all_statlogs is designed to be bound to an Hiera array containing all the possible statlogs.  It would be possible to instead populate it with all the statlogs actually present on the target node, as provided by a custom fact.

Alternatively, if you wrote the Statlog type as a Ruby plugin type, made it Ensurable, and made it perform prefetching, then you could do the purging part very simply via an instance of the Resources type.

Or if you were willing to put all wanted Services under Puppet management, then you could use a Resources to purge unmanaged Service resources, which would be along the lines of what you were thinking of doing.


John

[Vladimir Brik]

Thanks for the replies, everybody. They've been really helpful to clear-up my thinking!

Vlad