Git Repo Strategy

[Funsaized]

Hello,

I am relatively new to puppet and am trying to develop a good workflow in conjunction with git/github to keep a better version control system. The version of puppet that I am working with and has been implemented is a bit dated, and using R10k and developing a puppetfile would be quite time consuming. I know that R10k and dynamic environments is the recommended way of doing things, though for now I'm not sure if its the best for my scenario and how everything has been previously set up. My question is is there a simple way to just map one git repo for each environment (dev, QA, production, etc). That way changes could be made in the dev environment, then moved over to the correct repos when the changes are confirmed in order? Would this be as simple as declaring each folder withing the /puppet/environments folder as a git repo and controlling that way?

Deployment strategy

-       Upload changes to Dev repo

-       Deploy Dev changes to Dev master

-       Test

-       Merge Dev changes to QA repo

-       Rinse and repeat

[Christopher Wood]

On Fri, Jun 10, 2016 at 05:57:13AM -0700, Funsaized wrote:
> Hello,
>
> I am relatively new to puppet and am trying to develop a good workflow in
> conjunction with git/github to keep a better version control system. The
> version of puppet that I am working with and has been implemented is a bit
> dated, and using R10k and developing a puppetfile would be quite time
> consuming.

Yes it would be time consuming. It's worth the time. An upgrade would probably help you too.

> I know that R10k and dynamic environments is the recommended
> way of doing things, though for now I'm not sure if its the best for my
> scenario and how everything has been previously set up. My question is is
> there a simple way to just map one git repo for each environment (dev, QA,
> production, etc). That way changes could be made in the dev environment,
> then moved over to the correct repos when the changes are confirmed in
> order? Would this be as simple as declaring each folder withing the
> /puppet/environments folder as a git repo and controlling that way?

Been there, tried it, moved to r10k. The above is a great way to see your environments drift over time.

If you want each environment to be a separate git repo you can, but it sounds like you may as well have them all as branches in a single git repo. That way you can at least diff different branches easily and cherry pick single changes from one to the other.

> Deployment strategy
>
> -       Upload changes to Dev repo
>
> -       Deploy Dev changes to Dev master
>
> -       Test
>
> -       Merge Dev changes to QA repo
>
> -       Rinse and repeat

What is QA for, in this scenario? Determining that you've brought up the correct set of changes from dev? If so, why not use git itself to prove that you've brought up the correct changes, as specified by git commits, to your mainline branch?

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [1]puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> [2]https://groups.google.com/d/msgid/puppet-users/fe80ff27-af02-4437-bbc9-57c1cd56e5aa%40googlegroups.com.
> For more options, visit [3]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:puppet-users...@googlegroups.com
> 2. https://groups.google.com/d/msgid/puppet-users/fe80ff27-af02-4437-bbc9-57c1cd56e5aa%40googlegroups.com?utm_medium=email&utm_source=footer
> 3. https://groups.google.com/d/optout

[Rich Burroughs]

I'm assuming this could be done. We're talking about UNIX she'll commands and there's a way to do just about anything. But I can't imagine it being simple or fun to use. Like could you do Pull Requests on Github between these repos? Maybe, depending on how you set it up. People nowadays recommend against monolithic repos too, and that's what you'd have. You'd just have a bunch of them.

The normal recommended workflow with r10k is using branches for those environments, not separate repos. Then you have the ability to merge between branches, so it's easy to promote those changes along your pipeline.

I remember back before I started using r10k, it seemed very confusing to me. I think there's a bit more info out about it now. In terms of getting a Puppetfile setup, one of the hard things there is that you need to account for all of the dependencies. Rob Nelson made this cool Ruby gem that makes generating the file a bit easier. You can pass it a set of Forge modules and it will also include their dependencies:

https://github.com/rnelson0/puppet-generate-puppetfile/blob/master/README.md

It's pretty slick.

Personally I'd recommend you stick it out and figure out how to make r10k work for what you're doing. I would bet you'd be glad you did after. If you have problems ask specific question here or IRC or Slack. There are a lot of people using it now and there should be lots who can help.


Rich

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/fe80ff27-af02-4437-bbc9-57c1cd56e5aa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[broncosd183]

Awesome thanks for the feedback and options Rich and Christopher. I'm outlining a plan of attack now and going to make a pass at installing R10k and configuring it correctly. The main hurdle was the puppetfile and its dependencies; however, that looks much more feasible now.

[Bret Wortman]

I made the conversion a little over a year ago and it's been a dream ever since. The Puppetfiles aren't that hard -- We store each module in its own repo and use branches to determine environments. For each new environment we want to use, we just branch the "puppet" repo which contains the Puppetfile and let it know which modules will be under test for this environment. It's a lot simpler than it sounds.

[broncosd183]

Hey all,
I'm currently starting to implement the puppetfile format and have hit a wall of sorts. We currently are stuck on that old monolithic repo of modules and are eventually looking to move away from this sometime in the near future. My question is, for now is there any way to make a puppetfile for individual modules within this repo? We have hosted it on github and I understand how to pass the url and references if the modules are in their own repos. Can the same be done for modules in our monolithic repo? At the very least we were hoping to make a puppetfile for the current repo configuration and slowly transition out of it and update the puppetfile accordingly.

Thanks!

[broncosd183]

EDIT: I've found this link by Gary which details how to change the basemodulepath for each environment.conf file to effectively read in a monolithic repo containing all of the desired modules in your puppetfile  ( http://garylarizza.com/blog/2014/03/07/puppet-workflow-part-3b/ ).  My modified question is once this has been implemented, is there any way to implement a more precise module control in the puppetfile i.e pass references or commit tags if the modules had been in individual repos?

[Luke Bigum]

Git doesn't work like that, the repo in it's entirety is at a certain commit / tag / version, so you can't do what I think you are asking.

You might find this useful. We went from monolithic to split modules over a year ago, here's the Bash that did it. You'll want to adjust certain things, like the module name prefix:


#! /bin/sh

MODULE=$1
BASE_DIR=$PWD
WORK_DIR="${BASE_DIR}/working"

CONTROL_REPO='lmax-controlrepo'
GIT_REPO_BASE='https://GITLABSERVER/GITLABGROUP/'

cat << EOF
mkdir working
git clone ${GIT_REPO_BASE}/${CONTROL_REPO}.git ${WORK_DIR}/${CONTROL_REPO}
cd ${WORK_DIR}/${CONTROL_REPO}

git subtree split -P site/${MODULE} -b lmax-${MODULE}

cd ${WORK_DIR}
mkdir lmax-${MODULE}
cd lmax-${MODULE}
git init
git pull ${WORK_DIR}/${CONTROL_REPO} lmax-${MODULE}

git remote add origin ${GIT_REPO_BASE}/lmax-${MODULE}.git
git push origin -u master

cd ${BASE_DIR}
EOF



--
Luke Bigum
Senior Systems Engineer

Information Systems

>>>>> <https://groups.google.com/d/msgid/puppet-users/fe80ff27-af02-4437-bbc9-57c1cd56e5aa%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .

>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/4860f7e0-84f7-4627-b33e-722812c340c7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

---

LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN
http://www.LMAX.com/

Recognised by the most prestigious business and technology awards

2016 Best Trading & Execution, HFM US Technology Awards
2016, 2015, 2014, 2013 Best FX Trading Venue - ECN/MTF, WSL Institutional Trading Awards

2015 Winner, Deloitte UK Technology Fast 50
2015, 2014, 2013, One of the UK's fastest growing technology firms, The Sunday Times Tech Track 100
2015 Winner, Deloitte EMEA Technology Fast 500
2015, 2014, 2013 Best Margin Sector Platform, Profit & Loss Readers' Choice Awards

---

FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved.

This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). This message is not intended for any recipient(s) who based on their nationality, place of business, domicile or for any other reason, is/are subject to local laws or regulations which prohibit the provision of such products and services. This message is subject to the following terms (http://lmax.com/pdf/general-disclaimers.pdf), if you cannot access these, please notify us by replying to this email and we will send you the terms. If you are not the intended recipient, please notify the sender immediately and delete any copies of this message.

LMAX Exchange is the trading name of LMAX Limited. LMAX Limited operates a multilateral trading facility. LMAX Limited is authorised and regulated by the Financial Conduct Authority (firm registration number 509778) and is a company registered in England and Wales (number 6505809).

LMAX Hong Kong Limited is a wholly-owned subsidiary of LMAX Limited. LMAX Hong Kong is licensed by the Securities and Futures Commission in Hong Kong to conduct Type 3 (leveraged foreign exchange trading) regulated activity with CE Number BDV088.