Retrieve resource names of a given type without using the filesystem

[Bruno Bieth]

I'd like to do something like this


define mytype() {}

define delete_mytype() {}

mytype { 'a': }
mytype { 'b': }
mytype { 'c': }

$existing_mytypes = # something from facter

# but this doesn't work
$wanted_mytypes = Mytype <||>
 
Mytype<||> -> Delete_mytype<||>

delete_mytype { diff( $existing_mytypes, $wanted_mytypes ) }


I believe I could do something along those lines but it creates as many files as mytype invocation :


defined mytype() {
   # persist $name in the filesystem
}

class prepare_mytypes {
  # cleanup files created by previous invocation of mytype
}

Class['prepare_mytypes'] ->
  Mytype<||> ->
    Class['remove_old_mytypes']

class remove_old_mytypes {

  # look for files created by mytype
  # perform the same diff as explained above
}


Any idea?

[jcbollinger]

On Tuesday, May 20, 2014 2:41:41 AM UTC-5, Bruno Bieth wrote:

I'd like to do something like this


define mytype() {}

define delete_mytype() {}

mytype { 'a': }
mytype { 'b': }
mytype { 'c': }

$existing_mytypes = # something from facter

# but this doesn't work
$wanted_mytypes = Mytype <||>

No, it wouldn't.  I don't think collectors are rvalues, but if they were, they would probably evaluate to (1) an array of resource references, (2) in some unspecified order.  But the contents of a resource collection are evaluated very late, so as to ensure that all matching resources are included in it.  That's too late for the value to be used to declare other resources (else the purpose of late evaluation would not be served).

Additionally, facts are traditionally strings. In very recent Facter they can be arrays and hashes, too, but nowhere in there can you get resource references.  You can get resource names, however.

 

 
Mytype<||> -> Delete_mytype<||>

delete_mytype { diff( $existing_mytypes, $wanted_mytypes ) }


I believe I could do something along those lines but it creates as many files as mytype invocation :

I don't understand.  That is, of course you get one resource per declared Mytype instance -- that's what you specify by declaring those resources.  Why would you expect something different?

 

defined mytype() {
   # persist $name in the filesystem
}

class prepare_mytypes {
  # cleanup files created by previous invocation of mytype
}

Class['prepare_mytypes'] ->
  Mytype<||> ->
    Class['remove_old_mytypes']

class remove_old_mytypes {

  # look for files created by mytype
  # perform the same diff as explained above
}


Any idea?

So you are trying to remove unmanaged resources of the specified type?  Puppet has built-in support for that for plugin resource types that meet a couple of fairly simple criteria.  You would even dispense with any need for facter to be involved if you wrote your type as a (custom) type / provider pair.

On the other hand, if you must do this with defined types, then you can probably follow an approach similar to what you are attempting.

The main issue here is with attempting to glean from the catalog under construction what instances of Mytype have been declared.  It is always poor form to base catalog-building decisions on what has already been added to the catalog, as doing so introduces evaluation-order dependence into your manifest set.  Evaluation order is not specified, so evaluation-order dependence makes the contents of the constructed catalog incompletely defined and subject to unexpected change.

Instead of querying the catalog, you should know independently what resources of your type are intended to be present.  An array of their names would be suitable.  These days, the recommendation would probably be to maintain that data in Hiera.  You then use that both to declare the wanted resources and to determine (based on the data from Facter) what unwanted resources are present.  Puppet doesn't have a built-in diff() function, but it shouldn't be too hard to write a custom function to do what you need.  For prototyping, you could probably do it with a combination of inline_template() to perform the heavy lifting with split() to convert the template output from string to array.


John

[Bruno Bieth]

John, thanks for your detailed answer!

So you are trying to remove unmanaged resources of the specified type?  Puppet has built-in support for that for plugin resource types that meet a couple of fairly simple criteria.  You would even dispense with any need for facter to be involved if you wrote your type as a (custom) type / provider pair.

That seems to be the solution i'm looking for.

 

Instead of querying the catalog, you should know independently what resources of your type are intended to be present.  An array of their names would be suitable.  These days, the recommendation would probably be to maintain that data in Hiera.  You then use that both to declare the wanted resources and to determine (based on the data from Facter) what unwanted resources are present.  Puppet doesn't have a built-in diff() function, but it shouldn't be too hard to write a custom function to do what you need.  For prototyping, you could probably do it with a combination of inline_template() to perform the heavy lifting with split() to convert the template output from string to array.

I'm actually moving away from hiera. Just to give more context, I have a defined type to manage jenkins jobs that allows us amongst other things to
 - version jobs in a minimalistic fashion (instead of versioning xml).
 - build many similar projects by querying stash (using custom functions)
 
My defined type is called 'job_shell' and as I just said can be called multiple times in various places.

Bruno