Question about setting master-less server
35 views
Skip to first unread message
Hubert Lubaczewski
Mar 12, 2015, 1:58:14 AM3/12/15
to puppet...@googlegroups.com
Hi,
I'm trying to learn puppet by using it on a test machine I have. Figured that for single server, it makes sense to use master-less mode.
So, my question is like this. To set it up, I figured that:
1. /etc/puppet would be clone of some repo
2. in /etc/puppet/manifests/site.pp, I would add vcsrepo{} that would make sure that puppet will update itself on each run
3. I'll add a cronjob to periodically run "puppet apply /etc/puppet/manifests/site.pp"
Optionally, I would run "git pull" before actual puppet apply, so that puppet will run on already updated repo.
Does it make sense? Am I missing something? I know it's pretty basic, but in one place I had to write quite a lot of manifests/modules for puppet, and finally decided to setup whole machine, on my own, using puppet.
Thanks for any help/guidance,
depesz
I'm trying to learn puppet by using it on a test machine I have. Figured that for single server, it makes sense to use master-less mode.
So, my question is like this. To set it up, I figured that:
1. /etc/puppet would be clone of some repo
2. in /etc/puppet/manifests/site.pp, I would add vcsrepo{} that would make sure that puppet will update itself on each run
3. I'll add a cronjob to periodically run "puppet apply /etc/puppet/manifests/site.pp"
Optionally, I would run "git pull" before actual puppet apply, so that puppet will run on already updated repo.
Does it make sense? Am I missing something? I know it's pretty basic, but in one place I had to write quite a lot of manifests/modules for puppet, and finally decided to setup whole machine, on my own, using puppet.
Thanks for any help/guidance,
depesz
Charles Yeomans
Mar 12, 2015, 2:38:21 AM3/12/15
to puppet...@googlegroups.com
My current starter template for puppet projects is at https://github.com/declaresub/git-puppet-bootstrap . It may be useful for what you want to do. It is intended for use with VMs; I've tested it with Ubuntu 12.04 and Debian Wheezy on Linode and Vagrant VMs.
I have a simple puppet module for postgresql if you need one too :)
Charles
Hubert Lubaczewski
Mar 12, 2015, 3:46:47 AM3/12/15
to puppet...@googlegroups.com
Hi,
I checked the repo, but I don't understand what you wrote in the README. What is the purpose of these ssh keys? Why are you, in the run_puppet script, copying config to some place versioned by git commit, and run puppet from there, instead of just keeping /etc/puppet as git repo?
Basically - it is likely that your scripts solve problems that I am not aware of, yet, but then, would greatly appreciate more information what it is for - I can read what it does, but why?
depesz
I checked the repo, but I don't understand what you wrote in the README. What is the purpose of these ssh keys? Why are you, in the run_puppet script, copying config to some place versioned by git commit, and run puppet from there, instead of just keeping /etc/puppet as git repo?
Basically - it is likely that your scripts solve problems that I am not aware of, yet, but then, would greatly appreciate more information what it is for - I can read what it does, but why?
depesz
Jeff Adams
Mar 12, 2015, 1:18:28 PM3/12/15
to puppet...@googlegroups.com
We've settled on a git repo per module, then using r10k (thinking about
moving to librarian-puppet) to pull modules from git onto our nodes.
This happens in the wrapper script that we run out of cron that also
executes the puppet apply.
The advantage here is that if you happen to "break" puppet, you can
still get your fixes pulled onto the node without manual intervention.
- Jeff
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/b6da6bf0-9152-472b-b54f-85c0cf87b7d1%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/b6da6bf0-9152-472b-b54f-85c0cf87b7d1%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
________________________________
This message and any attached files contain confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or without error as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version.
moving to librarian-puppet) to pull modules from git onto our nodes.
This happens in the wrapper script that we run out of cron that also
executes the puppet apply.
The advantage here is that if you happen to "break" puppet, you can
still get your fixes pulled onto the node without manual intervention.
- Jeff
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/b6da6bf0-9152-472b-b54f-85c0cf87b7d1%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/b6da6bf0-9152-472b-b54f-85c0cf87b7d1%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
________________________________
This message and any attached files contain confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or without error as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version.
Charles Yeomans
Mar 12, 2015, 2:12:52 PM3/12/15
to puppet...@googlegroups.com
The basic idea is this: you have a server to be configured using puppet. On your machine, you have a git repository containing the puppet configuration, plus some other scripts. The puppet configuration sets up /var/lib/git as the home directory for user git, and a remote clone of your git repository is stored there. The script run_puppet.bash gets the puppet config from the git repository and copies it to /etc/puppet_hash, and sets a symlink to /etc/puppet.
The git repository on the server is bare, so I can't just make a symlink from /etc/puppet to the config directory in the repository. Thus the need to export it, and it seems a little wasteful to do the export every time. So I export to /etc/puppet_hash and symlink /etc/puppet to this directory. Then I export only if the repository has been updated.
You push from your git repository to the server repository over ssh; thus the purpose of the ssh keys is to enable this communication. It is often the case that puppet repositories contain sensitive information such as passwords, so I prefer not to put the repository on servers not under my control (i.e. github or bitbucket). Also, introducing something like github brings with it a fair amount of additional work to set up and maintain the chain. The approach I'm using works for multiple developers and multiple servers (e.g. development/test/production).
I suppose I could pull the sensitive bits out of the puppet configuration and store them in yet another location, but this starts up the Puppet Workaround Stack that I try to avoid.
Charles
> On Mar 11, 2015, at 11:46 PM, Hubert Lubaczewski <dep...@gmail.com> wrote:
>
> Hi,
> I checked the repo, but I don't understand what you wrote in the README. What is the purpose of these ssh keys? Why are you, in the run_puppet script, copying config to some place versioned by git commit, and run puppet from there, instead of just keeping /etc/puppet as git repo?
>
> Basically - it is likely that your scripts solve problems that I am not aware of, yet, but then, would greatly appreciate more information what it is for - I can read what it does, but why?
>
> depesz
>
> On Thursday, March 12, 2015 at 3:38:21 AM UTC+1, Charles Yeomans wrote:
> > On Mar 11, 2015, at 8:32 PM, Hubert Lubaczewski <dep...@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm trying to learn puppet by using it on a test machine I have. Figured that for single server, it makes sense to use master-less mode.
> >
> > So, my question is like this. To set it up, I figured that:
> > 1. /etc/puppet would be clone of some repo
> > 2. in /etc/puppet/manifests/site.pp, I would add vcsrepo{} that would make sure that puppet will update itself on each run
> > 3. I'll add a cronjob to periodically run "puppet apply /etc/puppet/manifests/site.pp"
> >
> > Optionally, I would run "git pull" before actual puppet apply, so that puppet will run on already updated repo.
> >
> > Does it make sense? Am I missing something? I know it's pretty basic, but in one place I had to write quite a lot of manifests/modules for puppet, and finally decided to setup whole machine, on my own, using puppet.
> >
> > Thanks for any help/guidance,
> >
> > depesz
> >
>
>
> I've used masterless puppet setups for several years.
>
> My current starter template for puppet projects is at https://github.com/declaresub/git-puppet-bootstrap . It may be useful for what you want to do. It is intended for use with VMs; I've tested it with Ubuntu 12.04 and Debian Wheezy on Linode and Vagrant VMs.
>
> I have a simple puppet module for postgresql if you need one too :)
>
> Charles
>
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/85ee6161-3430-4d60-8efc-6cbe66032508%40googlegroups.com.
The git repository on the server is bare, so I can't just make a symlink from /etc/puppet to the config directory in the repository. Thus the need to export it, and it seems a little wasteful to do the export every time. So I export to /etc/puppet_hash and symlink /etc/puppet to this directory. Then I export only if the repository has been updated.
You push from your git repository to the server repository over ssh; thus the purpose of the ssh keys is to enable this communication. It is often the case that puppet repositories contain sensitive information such as passwords, so I prefer not to put the repository on servers not under my control (i.e. github or bitbucket). Also, introducing something like github brings with it a fair amount of additional work to set up and maintain the chain. The approach I'm using works for multiple developers and multiple servers (e.g. development/test/production).
I suppose I could pull the sensitive bits out of the puppet configuration and store them in yet another location, but this starts up the Puppet Workaround Stack that I try to avoid.
Charles
> On Mar 11, 2015, at 11:46 PM, Hubert Lubaczewski <dep...@gmail.com> wrote:
>
> Hi,
> I checked the repo, but I don't understand what you wrote in the README. What is the purpose of these ssh keys? Why are you, in the run_puppet script, copying config to some place versioned by git commit, and run puppet from there, instead of just keeping /etc/puppet as git repo?
>
> Basically - it is likely that your scripts solve problems that I am not aware of, yet, but then, would greatly appreciate more information what it is for - I can read what it does, but why?
>
> depesz
>
> On Thursday, March 12, 2015 at 3:38:21 AM UTC+1, Charles Yeomans wrote:
> > On Mar 11, 2015, at 8:32 PM, Hubert Lubaczewski <dep...@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm trying to learn puppet by using it on a test machine I have. Figured that for single server, it makes sense to use master-less mode.
> >
> > So, my question is like this. To set it up, I figured that:
> > 1. /etc/puppet would be clone of some repo
> > 2. in /etc/puppet/manifests/site.pp, I would add vcsrepo{} that would make sure that puppet will update itself on each run
> > 3. I'll add a cronjob to periodically run "puppet apply /etc/puppet/manifests/site.pp"
> >
> > Optionally, I would run "git pull" before actual puppet apply, so that puppet will run on already updated repo.
> >
> > Does it make sense? Am I missing something? I know it's pretty basic, but in one place I had to write quite a lot of manifests/modules for puppet, and finally decided to setup whole machine, on my own, using puppet.
> >
> > Thanks for any help/guidance,
> >
> > depesz
> >
>
>
> I've used masterless puppet setups for several years.
>
> My current starter template for puppet projects is at https://github.com/declaresub/git-puppet-bootstrap . It may be useful for what you want to do. It is intended for use with VMs; I've tested it with Ubuntu 12.04 and Debian Wheezy on Linode and Vagrant VMs.
>
> I have a simple puppet module for postgresql if you need one too :)
>
> Charles
>
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/85ee6161-3430-4d60-8efc-6cbe66032508%40googlegroups.com.
hubert depesz lubaczewski
Mar 12, 2015, 2:42:36 PM3/12/15
to puppet...@googlegroups.com
I understand most of it, but why checkout as /etc/puppet_hash, and and not simply as /etc/puppet, and just `git pull` when necessary?
I.e. What is the benefit of having to do clone from scratch on every run (i.e. when something has changed)?
Charles Yeomans
Mar 12, 2015, 7:12:57 PM3/12/15
to puppet...@googlegroups.com
If I understand your question, the reason I don't just export to /etc/puppet is that the git repository contains files other than the puppet configuration, and I prefer to keep those files out of the puppet directory.
Certainly one could clone the bare repository in /var/lib/git, and symlink /etc/puppet to the puppet directory in that non-bare repository. Then the script that executes puppet apply would first run git pull, then puppet apply. I didn't do it that way for no reason other than I didn't think to do it that way. In fact it might be less complicated to implement, in exchange for some more disk space.
Charles
Certainly one could clone the bare repository in /var/lib/git, and symlink /etc/puppet to the puppet directory in that non-bare repository. Then the script that executes puppet apply would first run git pull, then puppet apply. I didn't do it that way for no reason other than I didn't think to do it that way. In fact it might be less complicated to implement, in exchange for some more disk space.
Charles
hubert depesz lubaczewski
Mar 12, 2015, 7:36:10 PM3/12/15
to puppet...@googlegroups.com
Why more disk space use? There would still be one clone of the repo, it just would be located in different place (not /etc/puppet_HASH, but directly /etc/puppet).
I think i can see one benefit of doing it the way you did it - we have guarantee that there will be no things in /etc/puppet that were added without adding them to git repo. Which is a good thing, I guess.Thanks,
depesz--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/GA4KHIebUc4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/190FCE9C-CD14-47C3-A338-D3ECEB13D6CB%40dakim.com.
Charles Yeomans
Mar 12, 2015, 10:09:18 PM3/12/15
to puppet...@googlegroups.com
No; there would be two repositories on the server. The repository in /var/lib/git is bare; recall that a bare git repository does not have a working copy. It would be possible to make the server repository non-bare, but that's generally a bad idea.
Charles
Charles
hubert depesz lubaczewski
Mar 12, 2015, 11:36:07 PM3/12/15
to puppet...@googlegroups.com
Sure. I wasn't talking about what's in /var/lib/git.
In your approach there is bare in /var/lib/git/server, and checkout/clone in /etc/puppet_HASH.Unless I am seriously missing something.
depesz--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/GA4KHIebUc4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/29040E55-23D7-46DA-9FA0-6C7163695842%40dakim.com.
Charles Yeomans
Mar 13, 2015, 1:59:44 AM3/13/15
to puppet...@googlegroups.com
In my current scheme, I export the git repository, grab the puppet subdirectory from the export, and move it to /etc/puppet_hash. I put it there instead of /etc/puppet because I only want to update the files if there is a new commit. To check this, I need to store the hash of the HEAD of the exported code somewhere. Instead of writing that information to a separate file, I stick it into the filename of the puppet config.
Charles
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKrjmhd00rWHZjUbMidyhDwv8hSabuTJUmAN_Q6oeT%3DtSqJa9g%40mail.gmail.com.
Charles
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKrjmhd00rWHZjUbMidyhDwv8hSabuTJUmAN_Q6oeT%3DtSqJa9g%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages