On code organization and the deprecation of include

[Andre Nathan]

Hello

I have a fairly large repository (~100 modules, ~50 classes, ~200 nodes). It is currently organized like this:

modules/

    apache2/

      manifests/

      files/

      templates/

    iptables/

      manifests/

      files/

      templates/

    postfix/

      manifests/

      files/

      templates/

    ...

manifests/

    classes/

        webserver.pp

        webserver/

            apache2.pp

            iptables.pp

        mta/

            postfix.pp

            iptables.pp

        ...

    nodes/

        webserver.pp

        webserver/

            web1.pp

            web2.pp

            ...

        mta.pp

        mta/

            mta1.pp

            mta2.pp

            ...

Subclasses in the modules directory are found via the autoloader, and as long as I follow the file naming conventions, that works fine. For classes and nodes, I have to use import. For example, in classes/webserver.pp I have "import webserver/*.pp" so that I can access classes webserver::apache2, webserver::iptables and so on. The same is done for nodes. I usually have a "generic node" where I set variables that are common for that class of nodes, and then the nodes themselves which inherit the generic node and set their own variables. To make that work, I use import in the same way as described above.

I understand that with the deprecation of "include", this will not work anymore. I can work around this for my classes by adding a new directory to the modulepath and then converting the directory structure to the manifests/files/templates structure with an init.pp file. However, what can I do about the nodes? The new feature of setting the manifest to a directory does not support subdirectories. Will I be forced to have all my nodes in the same directory and then rely on file names for organization? This is far from optimal in my opinion. What other alternatives do I have? If at all possible, I'd like to avoid using ENCs and keep using puppet files for node definitions.

Thanks in advance,

Andre

[Garrett Honeycutt]

Hi Andre,

I believe you mean the deprecation of 'import'. The easiest way to fix
this is to cat all of your files together into one site.pp file.

Your usage of a base node that other nodes inherit is an anti-pattern
though and will cause you grief as you grow. Recommend reading about
Craig Dunn's roles and profiles pattern and hiera_include for
classifying nodes.

Best regards,
-g

--
Garrett Honeycutt
@learnpuppet
Puppet Training with LearnPuppet.com
Mobile: +1.206.414.8658

[Andre Nathan]

On Wednesday, May 21, 2014 1:14:31 PM UTC-3, Garrett Honeycutt wrote:

I believe you mean the deprecation of 'import'.

Woops, indeed.

 

The easiest way to fix
this is to cat all of your files together into one site.pp file.

Now that's even worse than having all node files in a single directory.

 

Your usage of a base node that other nodes inherit is an anti-pattern
though and will cause you grief as you grow.

Why is it an anti-pattern? Isn't variable inheritance supported? Why can't I organize my nodes in sub-directories? It's something simple that greatly enhances code organization.

Best,

Andre

[Doug Forster]

It may be worth it to look more at hiera and adjust your codebase there.
For example my site.pp has one line: hiera_include('classes').

I organize my setup so that hiera looks for my nodes under hieradata/nodes/certname.yaml I see no reason not to allow further nesting if needed.

That being said it may be a good time to look into an ENC (External Node Classifier) like Foreman. I haven't done this yet as I work on a team and have to take baby steps so everyone keeps up.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/3be8fcd1-0626-4947-bf2b-d9d63475f470%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andre Nathan]

On Wednesday, May 21, 2014 1:59:09 PM UTC-3, Doug_F wrote:

I organize my setup so that hiera looks for my nodes under hieradata/nodes/certname.yaml I see no reason not to allow further nesting if needed.

It may be the only solution for me, but I'd rather not use YAML files as the risk of someone doing an edit and messing up with the indentation is high.

That being said it may be a good time to look into an ENC (External Node Classifier) like Foreman. I haven't done this yet as I work on a team and have to take baby steps so everyone keeps up.

This would remove the nodes from version control, which I'd rather not do either...

Thanks,

Andre

[jcbollinger]

On Wednesday, May 21, 2014 10:45:03 AM UTC-5, Andre Nathan wrote:

 

what can I do about the nodes? The new feature of setting the manifest to a directory does not support subdirectories. Will I be forced to have all my nodes in the same directory and then rely on file names for organization? This is far from optimal in my opinion. What other alternatives do I have? If at all possible, I'd like to avoid using ENCs and keep using puppet files for node definitions.

You can continue to use import for the time being, as deprecation is not the same as deletion.  Meanwhile, you can file a feature request for recursive traversal of the manifest directory, or for Puppet to support multiple manifest directories for a single environment.

Alternatively, you can coalesce each of your nodes/ subdirectories into a single file in the main manifest directory (e.g. webserver.pp containing your webserver base node plus all of your webN nodes).  That will leave you with a logical organization pretty similar to what you have now, and it won't clash with your local classes because you'll be moving those to modules (where really they should have been already).

Or you could move away from node blocks toward node classes, which would then be organized in a module instead of in your manifest directory.  There would be no particular reason to have any node blocks at all in that case; you could instead assign node classes via a top-level case statement or via hiera (or, indeed, via a simple ENC).


John

[jcbollinger]

Limited node inheritance is OK if you really understand what you're doing.  Most people don't: node inheritance doesn't work the way they expect.  The OO meaning of "inheritance" leads them in the wrong direction.

Multiple levels of node inheritance are a particularly precarious: in addition to inheritance likely not working as you expect or want, a node inheritance hierarchy locks you into a single principle node taxonomy.  Few real sites are well modeled by such a regime.


John

[Felix Frank]

On 05/21/2014 07:16 PM, Andre Nathan wrote:
> I organize my setup so that hiera looks for my nodes under
> hieradata/nodes/certname.yaml I see no reason not to allow further
> nesting if needed.
>
>
> It may be the only solution for me, but I'd rather not use YAML files as
> the risk of someone doing an edit and messing up with the indentation is
> high.

Note that YAML is not the only option. If you're afraid of indentation
mishaps, JSON may be a good alternative for you.

Regards,
Felix