Implementing hiera mapping of derived types to nodes

[Nathan Nobbe]

Hi,

I'm trying to implement a new feature of sorts and looking for some direction from folks who understand the puppet code. What I'm trying to do is create an ability to specify derived types via hiera much like hiera_include() works for classes. A quick aside, and the reason I'm interested in this to begin with, is most people I've asked suggest wrapping all node declarations in classes, however then you end up writing a bunch of classes that aren't really 'modules' in the practical sense of the term, they're really node definitions packaged as modules for the sake of declaration via hiera.

The idea here is in hiera, you can declare derived types for a node as such

# This will declare 2 file types for the node

hiera_file:

  /var/yum:

    ensure: directory

    owner: jenkins

    recurse: true

  /var/yum/custom-repo:

    ensure  : directory

    owner   : jenkins

    group   : jenkins

    require : "File['/var/yum']"

So thinking it would be cool to simply enumerate derived types in hiera I set off to write a module to implement the feature. I started with a class which basically looks like this

class type_mapper()

{

    $augeas = hiera_hash('hiera_augeas')

    if(!empty($augeas)) {

        create_resources(augeas, $augeas)

    }

    # similar block for each built-in derived type

}

But immediately realized dependency problems with type declarations that have 'require' et al. At first I thought I could write a function in ruby to deal with this, but then realized the complexity of the problem and I know there's code in puppet that handles this sort of dependency resolution already. I'm just looking around the source some and haven't found it yet. The function I'm trying to write would look something like this

module Puppet::Parser::Functions

  newfunction(:hiera_declare_types) do |args|

    # Raw type definitions provided by hiera

    # For now the hiera_hash() call is in RDL,

    # but that should move here too

    types = args[0]

    #------------------------------------------------------------

    # XXX What is the real puppet function for

    #     some_magic_puppet_dependency_reconciliation

    #------------------------------------------------------------

    ordered_types = some_magic_puppet_dependency_reconciliation(types)

    Puppet::Parser::Functions.function('create_resources')

    # Loop over all the type declarations

    ordered_types.each do |type, hash|

      # Loop over the declarations for a specific type

      hash.each do |title, real_hash|

        create_resources(type, hash)

      end

    end

  end

end

Is there such a function? I imagine there has to be because ultimately the type declarations are made one at a time, in some linear order. Generally speaking, do you see what I'm going for? Does it look like a cool addition for puppet? I feel like without something like this, using hiera causes you to move node definitions over to modules which isn't very attractive.

thanks,

-nathan

[Nathan Nobbe]

Ouch, looks like it was just erroneous ' characters in the required value. e.g.

{ required => "File['/etc/blah']" }

instead of

{ required => "File[/etc/blah]" }

now I see create_resources is working fine without any of the ordering logic I was talking about.

Had fun poking around the internals though :)

-nathan

[Nathan Nobbe]

Hi again,

I stayed up late last night and got this thing pretty much knocked out at the Ruby level. It's using the function_ paradigm to call other functions, like declare_resources and hiera_hash, but it works fine as is. I've uploaded the function as part of a forked hiera module on GitHub. The usage documentation is in the README.

I'm going to write up a blog post of the motivation behind this feature. I'll share it with you good folks and hope that you find interest in it. Maybe the blog post will provide persuasion, but I think something like this should be sitting in the standard tree next to hiera_include.rb :)

-nathan

On Friday, November 1, 2013 6:10:42 PM UTC-6, Nathan Nobbe wrote: