Git Repo Strategy

[Tom Tucker]

I just got back from PuppetConf last week and several presenters mentioned using more than one Git repo with Puppet.  Some even recommend having a repo per module.  For our initial Puppet deployment this seems a bit excessive.  My plan was to have three repos for each of our environments (Dev, QA and Production).  The contents of these repos would contain Puppet Enterprise directory of /etc/puppetlabs/puppet.

 

Deployment strategy

-       Upload changes to Dev repo

-       Deploy Dev changes to Dev master

-       Test

-       Merge Dev changes to QA repo

-       Rinse and repeat

 

 

Thoughts? Any tips for a Puppet and Git newbie in regards to file hierarchy, Git repo strategies, etc.

 

Thank you in advance,

 

Tom

 

Sample tree and repo of /etc/puppetlabs/puppet

# tree *

auth.conf

autosign.conf

console.conf   # File excluded this is site specific.  We will have a unique Puppet master for each env.

environments

├── development

│   ├── hieradata

│   │   └── environmentX.yaml

│   ├── manifests

│   │   └── site.pp

│   └── modules

└── production

fileserver.conf

hieradata

├── defaults.yaml

├── master.mydomain.com.yaml

└── production.yaml

hiera.yaml [error opening dir]

manifests

├── hieradata

│   └── hostgroups.yaml

└── site.pp

modules

├── custom

puppet.conf   # File excluded this is site specific.  We will have a unique Puppet master for each env.

puppetdb.conf   # File excluded this is site specific.  We will have a unique Puppet master for each env.

routes.yaml

ssl # Directory excluded this is site specific

< extra lines removed>

 

 

# cat hiera.yaml

---

:hierarchy:

    - "hieradata/fqdn/%{::fqdn}"

    - "%{environment}/%{::osfamily}"

    - "%{environment}/hieradata/%{::network_location}"

    - "%{environment}/hieradata/%{::systemrole}"

    - "hieradata/common"

 

:backends:

    - yaml

 

:yaml:

    :datadir: /etc/puppetlabs/puppet/environments

[Eric Shamow]

I highly recommend going with Gary Larizza’s functional puppet workflow unless you’ve got lots of prior experience. You can find part 1 here:

http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-1/

It’s a 3 part series, and there are good follow-ups on r10k and environments (http://garylarizza.com/blog/2014/03/26/random-r10k-workflow-ideas/) and an update to that post to cover directory environments (http://garylarizza.com/blog/2014/08/31/r10k-plus-directory-environments/)

Start there - there are criticisms to be had but it’s a good, tested functional workflow at use at a lot of orgs.

-Eric

-- 
Eric Shamow
Sent with Airmail

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAGymF1DKrTsh%2BNO%3DQLMpP1pM80ac3MMxvbo2p0aN9q9USXLj5Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Garrett Honeycutt]

On 9/30/14 7:17 PM, Tom Tucker wrote:
>
> I just got back from PuppetConf last week and several presenters
> mentioned using more than one Git repo with Puppet. Some even recommend
> having a repo per module. For our initial Puppet deployment this seems
> a bit excessive. My plan was to have three repos for each of our
> environments (Dev, QA and Production). The contents of these repos
> would contain Puppet Enterprise directory of /etc/puppetlabs/puppet.
>

Hi Tom,

One big repo full of modules stops scaling pretty quickly. This path
makes it impossible to work in scenario where you have multiple parties
using the same code base. Without individually versioning modules, you
will give up the opportunity to ever make a breaking change, because you
have to appease everyone. You also quickly get into situations where you
cannot deploy the newest version of the code because it changes multiple
modules and one of those causes issues. It leads to long running forks
that will never merge, cherry picking changes, and copy/pasting code.
Avoid this at all costs and save yourself the headaches.

Here's an example[1] of how I manage a bunch of modules, each in their
own repo. You could then use this Puppetfile with tools such as
librarian-puppet-simple, librarian-puppet, and r10k. I use this repo to
track all of my 'base' or 'common' modules and then use another repo to
track site specific modules. For that repo, I highly recommend r10k.

[1] - https://github.com/ghoneycutt/puppet-modules

Best regards,
-g

--
Garrett Honeycutt
@learnpuppet
Puppet Training with LearnPuppet.com
Mobile: +1.206.414.8658

[Jeremy T. Bouse]

I'll echo Garrett's comments that having all your modules in a single
repo severely hampers change. Like Garret I use a Puppetfile with which
I pull modules in from a a combination of the Puppet Forge and GitHub
repos. I do make mine available [1] as well and I include a README of
all the modules it includes and also note those that are pulled from the
Forge that are listed as "approved" or "supported".

I do use r10k and Puppet 3.7.x with directory environments so each
branch of my repo is handled as a separate environment when deployed by
r10k. I have my puppet environment setup so that all I do is commit to
the GitHub repo which triggers Jenkins CI jobs that validate and push
out to the puppet master. Once validated after being pushed out I can
promote my job and it is merged from one branch to the next and runs
through the cycle again.

Regard,
Jeremy

[Clayton O'Neill]

On Tuesday, September 30, 2014 7:17:51 PM UTC-4, Tom Tucker wrote:

I just got back from PuppetConf last week and several presenters mentioned using more than one Git repo with Puppet.  Some even recommend having a repo per module.  For our initial Puppet deployment this seems a bit excessive.  My plan was to have three repos for each of our environments (Dev, QA and Production).  The contents of these repos would contain Puppet Enterprise directory of /etc/puppetlabs/puppet.

I'll echo what other people have said here about single repo vs multiple repo.  You might be fine with a single repo if you have pretty basic configurations, but it has definite limits.  

One thing I'd *strongly* recommend against is having different repositories for different environments.  Assuming you're using Git, you should at most have these be separate branches in the same repository.  Longer term, you may want to move to using tags and (maybe) release branches for QA and Prod environments.  Having 3 branches will give you a lot more tools for moving changes between branches and for comparing differences between them.  Personally I think this is a lot more important than all modules in one repo vs a repo per module.