Puppet and SVN

[Alex Samad]

Hi

just starting out with puppet.

I found this http://projects.puppetlabs.com/projects/1/wiki/Puppet_Version_Control old page

talks about checking /etc/puppet into svn.

But on my centos install I have /etc/puppetlabs/puppet

do I add /etc/puppetlabs or /etc/puppetlabs/puppet to svn and if the later what about my codedir ?

Thanks

[Henrik Lindberg]

You probably want to use the project r10k to manage your
manifests/configurations that are stored in a source code control
repository.

Here is the page that explains a bit about r10k and SVN:
https://github.com/puppetlabs/r10k/blob/master/doc/dynamic-environments/svn-environments.mkd

There is also lots of material available on r10k itsef that is easy to
find if you google for it.

Regards
- henrik

> Thanks
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/ee63fdad-2d0f-4914-b9ca-0dbdb5044dc3%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/ee63fdad-2d0f-4914-b9ca-0dbdb5044dc3%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Alex Samad]

Ta

I was at the last puppet (sydney ) users group. heard r10k mentioned a lot.

Also I got the impression that git was more closely integrated with puppet.

Should I be l looking at git ?

A

> You received this message because you are subscribed to a topic in the

> Google Groups "Puppet Users" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/Q0ke0NzMRjY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/ac7d5df1-1798-e7d2-0215-b05166dc7b71%40puppet.com.

[Henrik Lindberg]

On 23/05/16 02:44, Alex Samad wrote:
> Ta
>
> I was at the last puppet (sydney ) users group. heard r10k mentioned a lot.
>
> Also I got the impression that git was more closely integrated with puppet.
>
> Should I be l looking at git ?
>

Absolutely. The only reason to use SVN would be if you are in an
organization where SVN is used everywhere, you know it well, or company
has a policy to only use SVN. (SVN is old school centralized source code
control management).

In every other case: use git.

You are also right in that git is more closely integrated with git, or
rather, git enables you to do certain things more easily. Parts of
puppet enterprise are more closely integrated with git.


- henrik

[Rob Nelson]

If you have a choice, you should almost always look at git over svn (IMO).

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PVrqBsaznadB9z9wTAS6s%3DdEHMZZpwNAqtCtYES7OMCyg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com

[Rilindo Foster]

You can use any VCS with Puppet. I know of one shop that uses Mercurial.

Most of the tooling and integration are closely coupled with git and most of the workflows assumes that you will be using git. So it would be a good idea to get up to speed with it.

Of note that you can use git with svn (which may be helpful if you are getting started:

https://git-scm.com/book/en/v1/Git-and-Other-Systems-Git-and-Subversion

- Rilindo

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PVrqBsaznadB9z9wTAS6s%3DdEHMZZpwNAqtCtYES7OMCyg%40mail.gmail.com.

[Alex Samad]

How hard is it to retro fit SVN / GIT onto a puppet install.

I am building from scratch, working through the doco.

Was thinking once I have a friendly setup then I would look at putting
that into SVN/GIT.

Thoughts are
install puppet
install puppetDB
install heira ??

setup environments, currently thinking
Prod - all prod env
SIM - testing for prod
INF - inf
NON Prod - anything thats not above
dev - testing
alex - personal

Then I was going to create a whole bundle of profiles break up into

OS app
things like
smtp
ssh
http
etc

have the default company setup

Company Apps
things that cover company apps


Then build some roles - based solely from profiles.

Then some how dynamically assign nodes to a roles / environment.


Sounds okay ??

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/00ed01d1b48c%24c1c09560%244541c020%24%40gmail.com.

[Christopher Wood]

On Mon, May 23, 2016 at 03:51:58PM +1000, Alex Samad wrote:
> How hard is it to retro fit SVN / GIT onto a puppet install.
>
> I am building from scratch, working through the doco.
>
> Was thinking once I have a friendly setup then I would look at putting
> that into SVN/GIT.

Do yourself a huge favour, use git as the tool you use to construct the setup. Use it from the start of the process, not just as what you put things into when you're done.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PV7FEU9tahp-5hSRqRLH4FXSD4o%3DkhejmjJ%3DoKr9%3DU_1g%40mail.gmail.com.

[Alex Samad]

Hi

Is it really that painful to retro fit git. or is the way you think
about the DB that different ?

Could you point a good starting point to read up on this ?

Thanks
Alex

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20160523211657.GA6621%40iniquitous.heresiarch.ca.

[Ramin K]

I have one of the largest SVN backed Puppet systems. It's painful,
slows us down, and we're getting off it this quarter. Finally.

If you don't have much dev experience, the idea that each branch exists
as a set of files in SVN initially sounds great. However the inability
to split work off into branches both short and long lived starts to take
its toll. Also tooling it hard, can't push a branch for someone else to
check out. You end up having "scratch space" for tests that eventually
have to be merged. It makes it hard to do sweeping changes as scratch
space has to encompass it all at least in most schemes I've observed.

I'm sure there are ways around some of the problems, but git based
branch development is ridiculously good with minimal investment. That
said git will seem like more work than you'd like when you have one or
two people working on the codebase. I never appreciated till I was on a
larger team.

Answering the other question, I recommend starting with
/etc/*puppetwhatever*/environments/{production|stage|etc}/ as releasses
of your src control rather than deploying directly to /etc/puppetlabs/
from src control.

Ramin

[Henrik Lindberg]

On 23/05/16 23:16, Christopher Wood wrote:
> On Mon, May 23, 2016 at 03:51:58PM +1000, Alex Samad wrote:
>> How hard is it to retro fit SVN / GIT onto a puppet install.
>>
>> I am building from scratch, working through the doco.
>>
>> Was thinking once I have a friendly setup then I would look at putting
>> that into SVN/GIT.
>
> Do yourself a huge favour, use git as the tool you use to construct the setup. Use it from the start of the process, not just as what you put things into when you're done.
>

+1

>> Thoughts are
>> install puppet
>> install puppetDB
>> install heira ??
>>

Since you are starting fresh. You are probably starting with something
like Puppet 4.4.x and since puppet 4.0.0 the packaging is "all in one"
for puppet. There is no need to install hiera separately.

- henrik

[Henrik Lindberg]

Alex, listen to Ramin!

Since I started coding, I have used (in order):

SCCS
RCS
CVS
ClearCase
Perforce
SVN
git

With the exception of git they have all been a total pain to use.

With git, being different from the others it may seem daunting at first.
Also, at the beginning your are bound to make beginner mistakes.
Take the advice to start early with git. Even if you are just
exploring/hacking/learning - that way if you screw things up it is not
the end of the world, and you learn the right way. Contrast that with
having to do everything with quality at some later point where you have
a working system and "retrofitting" - now you have to figure out all the
git related stuff at once without any prior experience.

Don't bother going first to SVN and then use the SVN to git. I have
migrated SVN to git - it works, but then you have to learn that domain
as well (it has its pitfalls).

As you go along you want to hang out on Slack or IRC (which ever suits
you best). There you will find many helpful colleagues that will help
you out, be it with puppet or git.

Best,
- henrik

[Christopher Wood]

It sounds like what you are thinking about is the roles/profiles paradigm, and your tool is r10k. Reading suggestions:

https://puppet.com/blog/git-workflows-puppet-and-r10k
http://somethingsinistral.net/blog/rethinking-puppet-deployment/
http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/
https://github.com/puppetlabs/r10k/blob/master/doc/dynamic-environments/workflow-guide.mkd

It will help you to download your first set of modules from the puppet forge. There's not much point re-implementing automated ntp management for instance. Later on you can do your own modules for any proprietary bits.

https://forge.puppet.com/

Think of it as using git as a tool to implement. You are not retrofitting git on your workflow, you are using git to record it.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PV3%2BTWUZMTvJFE_wBPM9m6eJsdLqSZKj0ZxxMFmqZgNWQ%40mail.gmail.com.

[Rob Nelson]

I wrote some articles on using Git with puppet and r10k. It's a little out of date in the referenced versions of puppet and r10k, so check to make sure you're using the modern file locations, but otherwise remains accurate. https://rnelson0.com/2014/05/19/puppet-and-git-201-r10k-setup-installation/

There's a link to a Git 100 series I wrote if you need assistance with Git itself.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PV3%2BTWUZMTvJFE_wBPM9m6eJsdLqSZKj0ZxxMFmqZgNWQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com

[Alex Samad]

Hi

Thanks for all the input.

What I have gathered

Start with GIT from the beginning. Okay I can do that

/etc/*puppetwhatever*/environments/{production|stage|etc}/ is good

Look at using / implementing r10k


Profiles / roles good.

No need to reinvent modules - wasn't actually planning on that.

From the puppet user group meeting I got that it is good to (example ntp)

1) reuse forge ntp module
2) wrap it up with $job defaults
3) use the new ntp profile


Something i was looking at doing was managing the server firewall.

I was going to produce some sort of a template with places in it where
I can add dynamic content - depending on what apps where installed on
the server.


Example
profile - ABC Firewall
this would be basic

allow related,connected
allow ssh
<Placeholder for access from any where>
reject anything not from company ip address
<Placeholder for access from inside where>
drop broadcast
drop multicast
allow zabbix monitoring
log with limit
reject with limit
drop


so if I applied this profile to server A it would get the basic firewall.

The next step is if I had profile "app A"
if it had something like
rules to add to from anywhere
allow inbound port 80
allow inbound port 443

rules to add to only from local network
allow inbound port 8080


I would hope to get puppet to update the basic firewall with info from
app A and other profiles.


Is this possible ?

A

> https://groups.google.com/d/msgid/puppet-users/CAC76iT-TgCx0KbgavCwjXO8Z-J6xpsCBVn5ntueWRUw3wjpWrw%40mail.gmail.com.

[Rob Nelson]

Yep! I do that with some profiles:

Base https://github.com/puppetinabox/controlrepo/blob/production/dist/profile/manifests/base.pp calls linuxfw and its pre/post subclasses https://github.com/puppetinabox/controlrepo/blob/production/dist/profile/manifests/linuxfw.pp https://github.com/puppetinabox/controlrepo/tree/production/dist/profile/manifests/linuxfw to set up the genera rules all bodes have. 

Component profiles like Apache allows port 80 https://github.com/puppetinabox/controlrepo/blob/production/dist/profile/manifests/apache.pp

The names of the firewall rule resources are sorted numerically, so you can have multiple 100 rules in different profiles and it won't cause a problem unless one of them is a deny, in which case you probably want to start the name with 99 - or have unique rules across profiles, though that doesn't scale very well. 

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PX9q%2BntHsAXaqWM%2BrXNxJJycdauMhJXDLOpkZ390O75WA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com

[Alex Samad]

Hi

Cool, so I presume the module firewall is from the forge
https://github.com/puppetlabs/puppetlabs-firewall this one ?

I notice you have your modules / code on github aren't you concerned
about having it in a public place ?

A

> https://groups.google.com/d/msgid/puppet-users/CAC76iT95uN7J_oDb4yF7h%3D41sD%3DpDfZdShw046O8%3DTrsVVA_zg%40mail.gmail.com.

[Rob Nelson]

Yes, you can see the modules used in the Puppetfile.

That's a learning tool / reference architecture, suitable for a quick lab setup. My personal and work setups are not public, though.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PUpjnY3i1YnpOAtmCM6amGLc3QgyOyVXkp2qVVnVXOZrw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com

[Alex Samad]

Hi


Finally got some time to look at this

On 24 May 2016 at 11:22, Rob Nelson <rnel...@gmail.com> wrote:

> I wrote some articles on using Git with puppet and r10k. It's a little out
> of date in the referenced versions of puppet and r10k, so check to make sure
> you're using the modern file locations, but otherwise remains accurate.
> https://rnelson0.com/2014/05/19/puppet-and-git-201-r10k-setup-installation/
>
> There's a link to a Git 100 series I wrote if you need assistance with Git
> itself.

I notice in the previous installs of puppet every thing was under
/etc/puppet and you recommended putting that under git.


I see with the latest version of puppet I have

/etc/puppetlabs
should I place that under git

The only thing of concern there would be the ssl directory.

or do i just add
code/
puppet/
puppetserver/


THanks
Alex

[Rob Nelson]

Alex, some of the later r10k articles have changed workflows, you may want to read the whole series before implementing anything. I highly recommend you check out Gary's Workflows Evolved article at http://garylarizza.com/blog/2015/11/16/workflows-evolved-even-besterer-practices/. The third link is to a reference controlrepo that Puppet suggests be used as a basis for building your own controlrepo (https://github.com/puppetlabs/control-repo). The controlrepo is what r10k would be deploying.

You may also want to capture other items, mostly for bootstrapping, such as .pp files to configure hiera and r10k. Whether you place those in your controlrepo or another repo is up to you.

Rob Nelson
rnel...@gmail.com

Alex

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PUs%3Dr6a74VPQ9K0z40Uu_bHXAdfeJEYMph4JCJ%3DRcFLQw%40mail.gmail.com.

[Alex Samad]

Thanks. I did get onto the code control section which seems to
suggest I should use it instead of git/r10k as it hids this away
behind the scene !

> You received this message because you are subscribed to a topic in the

> Google Groups "Puppet Users" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/Q0ke0NzMRjY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> puppet-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/CAC76iT9ANbHm59X_fLcsGYxERgFBBHC97bk2yQR9EvirFWtz0Q%40mail.gmail.com.

[Rob Nelson]

It does not hide it, but it does automate some of the installation and setup. You'll still need to live in Git, and should at least be familiar with using r10k even if you set up a webhook and don't run it manually all the time. It's what you use to rescue yourself when you mess up, as I can attest to!

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PUr46gus%2Bipc8H%3DsOAC%2Bzh7NXHOqvDCudJnqhDK6O9jjw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com

[Alex Samad]

Hi

Thanks for this again, its has helped allot.

Question would it be fair to say you can't do firewall rules from hiera, that they have to be associated with a module..

Alex

>> >> >>> send an email to puppet-users+unsubscribe@googlegroups.com
>> >> >>> <mailto:puppet-users+unsub...@googlegroups.com>.

>> >> >>> To view this discussion on the web visit
>> >> >>>
>> >> >>>
>> >> >>> https://groups.google.com/d/msgid/puppet-users/ee63fdad-2d0f-4914-b9c
>> >> >>> a-0dbdb5044dc3%40googlegroups.com
>> >> >>>
>> >> >>>
>> >> >>> <https://groups.google.com/d/msgid/puppet-users/ee63fdad-2d0f-4914-b9ca-0dbdb5044dc3%40googlegroups.com?utm_medium=email&utm_source=footer>.
>> >> >>> For more options, visit https://groups.google.com/d/optout.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >>
>> >> >> Visit my Blog "Puppet on the Edge"
>> >> >> http://puppet-on-the-edge.blogspot.se/
>> >> >>
>> >> >> --
>> >> >> You received this message because you are subscribed to a topic in
>> >> >> the
>> >> >> Google Groups "Puppet Users" group.
>> >> >> To unsubscribe from this topic, visit
>> >> >>
>> >> >> https://groups.google.com/d/topic/puppet-users/Q0ke0NzMRjY/unsubscribe.
>> >> >> To unsubscribe from this group and all its topics, send an email to

>> >> >> puppet-users+unsubscribe@googlegroups.com.

>> >> >> To view this discussion on the web visit
>> >> >>
>> >> >> https://groups.google.com/d/msgid/puppet-users/ac7d5df1-1798-e7d2-0215-b05166dc7b71%40puppet.com.
>> >> >>
>> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups "Puppet Users" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,

>> >> > send an email to puppet-users+unsubscribe@googlegroups.com.

>> >> > To view this discussion on the web visit
>> >> > https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PVrqBsaznadB9z9wTAS6s%3DdEHMZZpwNAqtCtYES7OMCyg%40mail.gmail.com.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> > --
>> >> > You received this message because you are subscribed to a topic in
>> >> > the Google Groups "Puppet Users" group.
>> >> > To unsubscribe from this topic, visit
>> >> > https://groups.google.com/d/topic/puppet-users/Q0ke0NzMRjY/unsubscribe.
>> >> > To unsubscribe from this group and all its topics, send an email to

>> >> > puppet-users+unsubscribe@googlegroups.com.

>> >> > To view this discussion on the web visit
>> >> > https://groups.google.com/d/msgid/puppet-users/00ed01d1b48c%24c1c09560%244541c020%24%40gmail.com.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups "Puppet Users" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send

>> >> an email to puppet-users+unsubscribe@googlegroups.com.

>> >> To view this discussion on the web visit
>> >> https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PV7FEU9tahp-5hSRqRLH4FXSD4o%3DkhejmjJ%3DoKr9%3DU_1g%40mail.gmail.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to a topic in the
>> > Google Groups "Puppet Users" group.
>> > To unsubscribe from this topic, visit
>> > https://groups.google.com/d/topic/puppet-users/Q0ke0NzMRjY/unsubscribe.
>> > To unsubscribe from this group and all its topics, send an email to

>> > puppet-users+unsubscribe@googlegroups.com.

>> > To view this discussion on the web visit
>> > https://groups.google.com/d/msgid/puppet-users/20160523211657.GA6621%40iniquitous.heresiarch.ca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to puppet-users+unsubscribe@googlegroups.com.

>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PV3%2BTWUZMTvJFE_wBPM9m6eJsdLqSZKj0ZxxMFmqZgNWQ%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
>
> Rob Nelson
> rnel...@gmail.com
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/Q0ke0NzMRjY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> puppet-users+unsubscribe@googlegroups.com.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/CAC76iT-TgCx0KbgavCwjXO8Z-J6xpsCBVn5ntueWRUw3wjpWrw%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PX9q%2BntHsAXaqWM%2BrXNxJJycdauMhJXDLOpkZ390O75WA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com