Profile as a Git Sub-module

[Fraser Goffin]

Been using the Roles and Profiles pattern for a while, but only recently had the need to create a Profile that is required in a number of Roles.

Typically we use Git sub-modules when we want to do this sort of thing, but I don't want to make this a discussion about Git sub-modules vs. other constructs, but rather ask about the effect of doing so iro Puppet, and whether the solution described here seems reasonable (or not).

The problem (if indeed there is one) arises because of the structure needed to create a Git repo and the 'normal' structure of Profile classes, or at least how they are generally shown in examples. Like so :-

./module/profiles/manifests/my_profile1.pp

Note the class my_profile1 exists directly within the manifests folder of profiles and might look like this :-

class profiles::my_profile1 {

  include my_module1
  include my_module2
  ...
}

... and if I wanted to use that classes from a 'role' I might use something like :-

class roles::my_role {

  include profiles::my_profile1

  ... include other profiles
}

However, when I create a repo in Git, I would want that repo to be called (in this case) 'my_profile1', and, when I add it as a sub-module, that would be the folder name that is created, which clearly doesn't follow the folder structure above. Hmmm

Of course profile classes don't *have* to be placed as I have shown, so, I *could* create a folder called 'my_profile1' *below* profile/manifests, but then I would need to decide what to call that class and I would also need to reference it differently from the 'role'.

For example, I *could* call the class 'impl' (not very imaginative I know), so I would end up with :-

./module/profiles/manifests/my_profile1/impl.pp

containing :-

class profiles::my_profile1::impl {

  include my_module1
  include my_module2
  ...
}

and called from the 'role' class, thus :-

class roles::my_role {
 
  include profiles::my_profile1::impl

  ... include other profiles
}

This is just a first attempt and it works and doesn't seem too bad (at least in the sense that the structure is reasonably intuitive and the profile class name is fairly descriptive of its intent). It might get a bit clumsy if I wanted to create further sub-classes of the main profile to break-out functionality from 'impl', but thats perhaps unlikely. There do also appear to be some advantages over dumping all profile classes in a single folder (such as ... it's cleaner to add [say] an *examples* folder within my_profile1 for testing, thus keeping everything related to this class together, ... ).

But is there another (perhaps better) approach that achieves the same thing (allows a profile class to be shared amongst roles and stored in Git as an independantly versioned repo and referenced as a sub-module) ?

Kind Regards

Fraser.

[Christopher Wood]

Below is all really complicated and I keep losing track of what is going on.

We have all profile classes in a single puppet module (in git). Then in a role class I would:

include ::profile::common
include ::profile::this
include ::profile::other::that

I presume if somebody had a Super Special Flower sort of "business need" (hasn't happened yet) I'd have them start up another profile module, and then their role would look like this:

include ::profile::common
include ::ssf::this
include ::ssf::that

Other than that I don't really see what advantage you get from git submodules as opposed to r10k. You can include a class from any profile-level module and that will probably work for you. Would you mind expounding on what git submodules gets you here?

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [1]puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> [2]https://groups.google.com/d/msgid/puppet-users/e062e2f4-15ee-4477-81aa-bb9465ebcbef%40googlegroups.com.
> For more options, visit [3]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:puppet-users...@googlegroups.com
> 2. https://groups.google.com/d/msgid/puppet-users/e062e2f4-15ee-4477-81aa-bb9465ebcbef%40googlegroups.com?utm_medium=email&utm_source=footer
> 3. https://groups.google.com/d/optout

[Fraser Goffin]

I guess having a single module containing all possible profile classes is one way to go. So in this case you just have a single Git repo called profiles, right, and as/when the need for a new combination of resource modules arises, would just add a new class for that purpose to this module ?

I suppose I feel like I want to manage the life-cycle of each unique instance independently from all of the others, it's external dependencies, test framework etc ... rather than bundling them all together into an über-module. There is not necessarily a strong relationship (cohesion) between these classes other than that their implementation style (composition of other modules). It seems to me to better promote separation of concerns to keep them apart but I accept that might be an illusion. At compile time, I assume that the catalogue only includes those classes that the role explicitly declares, so there isn't any issue with dependencies on classes that are in the module but not used ... is that correct (if not, that would certainly be a case for separation imo) ?

It's possible that the number of profile classes *could*, over time, become large, and thus have an impact on change management. Also from a unit of deployment perspective, I'm more attracted to the idea of not including more code (classes) than I really need, which would always be the case using the single profile module approach. Again, not an especially strong argument against your proposal given that we routinely bundle 3rd party libraries with our applications when only using a tiny fraction of the available classes and methods.

Hopefully you can see some of my thought process and, to answer your question, why I chose to experiment using a Git sub-module per profile.

But you have given me another model to consider, and that was what I asked for, so thank you, I appreciate you taking the time.

Kind Regards

Fraser.

[Luke Simmons]

I was surfing, trying to find different forums where this type of case popped up, and ran into your discussion. I even posted to the Puppet forums without any answers. I did though run into a nice section in the comments here:

http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/

The entire post is certainly worth a read, but the comments provide a Q&A about how to deal with profiles needing to be maintained in separate repos, which sounds, more or less, like what you're trying to do.

[Fraser Goffin]

Thanks Luke. I guess like anyone using Puppet, Gary is always a good starting point (I have certainly quoted some of his material in practice presentations I have given internally). I had read this one including the Q&A around roles and profiles, but it was so long ago that I had forgotten, so thanks for reminding me.

I still haven't made my mind up about what should be our preferred approach, but, in true agile style I'm following the simplest working solution first off and then refactoring.

Actually I'm spending most of my time docker'izing these days and figuring out where Puppet fits in (and trying to convince those who would write all of their provisioning using just Docker's own DSL (aka: bash :-) that there is another way. Not sure I'm going to win yet though !

Regards

Fraser.

[Luke Simmons]

Always a smart move to start simple...

For what it's worth, a large company here in western Sweden I know has started using Ansible and Docker and then simply swapping out docker instances for upgrades. I could imagine something similar could be accomplished in Puppet as well.

Btw, good luck fighting the good fight! It's tough to bash the bash out of the Unix guys!