Puppet, git & security

[Alex Harvey]

Hi all,

In my company we have a security policy that frowns upon things like puppet masters making git pull requests to other network segments.  Allowing code to be pushed into these segments is less of a problem.

This policy makes it difficult to do stuff like,
https://puppetlabs.com/blog/git-workflow-and-puppet-environments/

I am wondering if anyone out there has ever faced a similar problem and has worked out a way to build a push-only configuration from GIT code repositories to puppet masters.

Best regards,
Alex

[devzero2000]

Sorry for the top posting.

Imho, i think this is a question that could be asked on the git mailing list.

Best

2013/5/15, Alex Harvey <alexh...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users...@googlegroups.com.
> To post to this group, send email to puppet...@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

--
Inviato dal mio dispositivo mobile

[Denmat]

I haven't worked out a pure git way but Jenkins, git export, rsync are a good solid combo :)

[Alex Harvey]

On Wednesday, May 15, 2013 3:40:28 PM UTC+10, denmat wrote:

I haven't worked out a pure git way but Jenkins, git export, rsync are a good solid combo :)

Do you know of any documentation or blog posts from others using a configuration like this?  My initial thinking was to use rsync but I am concerned about getting bitten by Puppet manageability or scalability issues down the track that I haven't thought of.  Thus my interest in the experiences of others who may have a push-only relationship between their revision control systems and the puppet masters.

[Alex Harvey]

On Wednesday, May 15, 2013 2:51:14 PM UTC+10, yersinia.spiros wrote:

Sorry for the top posting.

Imho, i think this is a question that could be asked on the git mailing list.

Sorry, my question apparently isn't clear enough - this definitely isn't a git question that can be answered at the git mailing list.  I am interested in both how you might configure a push-only solution from revision control system to puppet masters, but more importantly, can Puppet be just as easy to manage if I do this?

[Stephen Gran]

Your push server can run git update and then rsync to the masters. If
you rsync the entire git tree, your workflow remains the same as in the
article, except that the transport for getting the git repo onto the
masters has changed.

Cheers,
--
Stephen Gran
Senior Systems Integrator - guardian.co.uk
Please consider the environment before printing this email.
------------------------------------------------------------------
Visit guardian.co.uk - website of the year

www.guardian.co.uk www.observer.co.uk www.guardiannews.com

On your mobile, visit m.guardian.co.uk or download the Guardian
iPhone app www.guardian.co.uk/iphone and iPad edition www.guardian.co.uk/iPad

Save up to 32% by subscribing to the Guardian and Observer - choose the papers you want and get full digital access.
Visit guardian.co.uk/subscribe

---------------------------------------------------------------------
This e-mail and all attachments are confidential and may also
be privileged. If you are not the named recipient, please notify
the sender and delete the e-mail and all attachments immediately.
Do not disclose the contents to another person. You may not use
the information for any purpose, or store, or copy, it in any way.

Guardian News & Media Limited is not liable for any computer
viruses or other material transmitted with or as part of this
e-mail. You should employ virus checking software.

Guardian News & Media Limited

A member of Guardian Media Group plc
Registered Office
PO Box 68164
Kings Place
90 York Way
London
N1P 2AP

Registered in England Number 908396

[jcbollinger]

If you want your masters to rely on a revision control system for manifests, data, or whatever, then it follows that the masters must be permitted to access that system.  If they may not do so across network segments, then it follows that the repository must be hosted on the same segment as the masters.  It also follows that if the masters are on different segments then you need some kind of cross-segment replication of the revision-control repository.

Furthermore, git is probably not the best choice of revision-control system for what I think you're asking.  As I understand git operation (which is imperfectly), you could certainly push changes to a remote repository, but you would also need to pull from the repository side before those changes would be available to other clients (such as the masters).  This is different from the more traditional approach of systems such as Subversion.

As far as how easy it is to manage your Puppet infrastructure, I don't think I understand how you're partitioning subsystems.  I would account everything you're talking about as part of managing Puppet, and clearly, operating under the constraints you describe is harder than operating without them.  On the other hand, if you postulate that the problem of getting the needed code and data into an accessible repository is solved and out of scope, then why would you suppose there would be any difference between managing Puppet in your environment and managing it in a less-constrained one?


John

[Alex Harvey]

On Thursday, May 16, 2013 12:48:22 AM UTC+10, jcbollinger wrote:

If you want your masters to rely on a revision control system for manifests, data, or whatever, then it follows that the masters must be permitted to access that system.  If they may not do so across network segments, then it follows that the repository must be hosted on the same segment as the masters.  It also follows that if the masters are on different segments then you need some kind of cross-segment replication of the revision-control repository.

Furthermore, git is probably not the best choice of revision-control system for what I think you're asking.  As I understand git operation (which is imperfectly), you could certainly push changes to a remote repository, but you would also need to pull from the repository side before those changes would be available to other clients (such as the masters).  This is different from the more traditional approach of systems such as Subversion.

As far as how easy it is to manage your Puppet infrastructure, I don't think I understand how you're partitioning subsystems.  I would account everything you're talking about as part of managing Puppet, and clearly, operating under the constraints you describe is harder than operating without them.  On the other hand, if you postulate that the problem of getting the needed code and data into an accessible repository is solved and out of scope, then why would you suppose there would be any difference between managing Puppet in your environment and managing it in a less-constrained one?

John, thanks for your comments.

I was actually thinking of doing something similar to what Stephen Gran suggested above; let rsync can take care of ensuring that all puppet masters always have the same copy of the same code tree.  So in that situation there is no need for the puppet masters to have access to the revision control system.  Indeed, I am not even sure that this configuration will be any more troublesome than having the puppet masters contact the git repositories using git pulls.

What do you think?

[jcbollinger]

On Wednesday, May 15, 2013 9:20:37 PM UTC-5, Alex Harvey wrote:

I was actually thinking of doing something similar to what Stephen Gran suggested above; let rsync can take care of ensuring that all puppet masters always have the same copy of the same code tree.  So in that situation there is no need for the puppet masters to have access to the revision control system.  Indeed, I am not even sure that this configuration will be any more troublesome than having the puppet masters contact the git repositories using git pulls.

What do you think?

I think it's workable.   Implications to consider include:

• the model only makes sense if you have multiple independent contributors updating the central git repo.  If there were only one, then you would probably be better off to cut out the middleman and just have that one perform the rsync.
• Git still may not be the best source-control system for this purpose, as you still need action on the side of the central repo to incorporate changes pushed by remote clients.  From time to time you will probably need to resolve conflicts there, as well.  (As opposed to some other systems, in which conflicts are resolved by committers.)
  
• Masters may not always be working with the most up to date manifests, and at any given time, different masters may be working with different manifest sets.

None of those seem like deal-breakers to me.


John

[Martin Langhoff]

On Wed, May 15, 2013 at 2:44 AM, Stephen Gran
<stephe...@guardian.co.uk> wrote:
> Your push server can run git update and then rsync to the masters.

Why rsync if you have git?

You have

- the machine(s) where you edit and make commits on git, you then
_git push_ to what I'll call a "gold" git server

- the gold git server, normally it answers requests, but in Alex's
company it would run a cron to push to essentially local mirrors in
their various networks.

- "local puppet masters" -- given that you are using push from the
gold server to the "local puppet masters", you can use a post-receive
hook to auto-checkout the update as part of the push. so as soon as
the push completes, the puppet server code is serving the new files...

- puppet clients that pull from their local puppet masters.

--

if you use rsync, you have a potentially large window of time during
which the tree of files is inconsistent on the local puppet masters --
larger if the network connection is slow or unreliable. rsync updates
each file atomically, but not the tree atomically.

Using git shrinks that window considerably -- but not completely.

I am currently working on a similar infra, but local puppet masters
are allowed to "git fetch" from the gold server. The tooling I've
developed around it is here http://repo.or.cz/w/puppet-git.git/ with
some discussion of usage and design here
https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/OilxMytnD_k

cheers,



m
--
martin....@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff

[Alex Harvey]

On Friday, May 17, 2013 12:35:24 AM UTC+10, Martin Langhoff wrote:

On Wed, May 15, 2013 at 2:44 AM, Stephen Gran
<stephe...@guardian.co.uk> wrote:
> Your push server can run git update and then rsync to the masters.

Why rsync if you have git?

Martin, John, thanks for your feedback - it's very helpful.