How To Expand an .erb Template Without Doing a Full Puppet Run?
Jon Forrest
of you do. We use git to maintain these files, which are
then pulled by the puppet master machine. These zone files are
actually puppet template .erb files.
The other day, I made a mistake which resulted in some zone
files with syntax errors in them. Puppet faithfully distributed
the erroneous zone files to the name servers, which killed them.
To prevent this from happening again, I want to be able to
somehow expand the template files and run named-checkzone
as part of a git pre-commit hook. I want this expansion to
occur without making a normal puppet run so that the bad
files won't get distributed.
Can anybody point me to a technique for doing this?
Cordially,
Jon Forrest
Nigel Kersten
Cordially,
Jon Forrest
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
Product Manager, Puppet Labs
Jonathan Stanton
> On Sat, Sep 10, 2011 at 5:36 PM, Jon Forrest <nob...@gmail.com> wrote:
>
>> We use puppet to distribute named zone files, like many
>> of you do. We use git to maintain these files, which are
>> then pulled by the puppet master machine. These zone files are
>> actually puppet template .erb files.
>>
>> The other day, I made a mistake which resulted in some zone
>> files with syntax errors in them. Puppet faithfully distributed
>> the erroneous zone files to the name servers, which killed them.
>>
>> To prevent this from happening again, I want to be able to
>> somehow expand the template files and run named-checkzone
>> as part of a git pre-commit hook. I want this expansion to
>> occur without making a normal puppet run so that the bad
>> files won't get distributed.
>>
>> Can anybody point me to a technique for doing this?
>>
>
>
> git cat-file blob :0:$indexfile | erb -x -T - | ruby -c 2> $error_msg >
> /dev/null ;
>
> from:
>
> http://projects.puppetlabs.com/projects/1/wiki/Puppet_Version_Control
>
> should get you going.
>
Maybe I'm missing something here, but I think Jon was asking something a bit different -- he doesn't want to check the validity of the erb template (i.e. ruby syntax check) but syntax check the named zone file generated by the template.
So the tricky bit is how to get the variables out of the puppet manifests that the erb template needs to generate the output file that 'would' be generated by a new puppet run for this node -- without the actual puppet run (as he asks at the end of the email).
My first thought is that the only accurate way to do this is by doing a full puppet run, as any part of the node's manifest could effect the variables used in the zone file template. You should be able to get away with a --noop run so the changes won't actually be applied (because noop does generate files from templates, but you would need to have a way to capture the newly generated zone file on the client host and run the named-checkzone there.
Cheers,
Jonathan
-------------------------------------------------------------------------------
Jonathan Stanton jona...@spreadconcepts.com
Spread Group Messaging www.spread.org
Spread Concepts LLC www.spreadconcepts.com
-------------------------------------------------------------------------------
>
>
>
>>
>> Cordially,
>> Jon Forrest
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet...@googlegroups.com.
>> To unsubscribe from this group, send email to puppet-users+unsubscribe@**
>> googlegroups.com <puppet-users%2Bunsu...@googlegroups.com>.
>> For more options, visit this group at http://groups.google.com/**
>> group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en>
>> .
>>
>>
>
>
> --
> Nigel Kersten
> Product Manager, Puppet Labs
>
> *Join us for **PuppetConf *
> <http://www.bit.ly/puppetconfsig>
> Sept 22/23 Portland, Oregon, USA.
> *
> *
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
Jon Forrest
>
> Maybe I'm missing something here, but I think Jon was asking
> something a bit different -- he doesn't want to check the validity of
> the erb template (i.e. ruby syntax check) but syntax check the named
> zone file generated by the template.
Precisely. Maybe later I'll face the issue of ruby syntax
problems but right now I need to detect named syntax errors
before they cause problems.
> So the tricky bit is how to get the variables out of the puppet
> manifests that the erb template needs to generate the output file
> that 'would' be generated by a new puppet run for this node --
> without the actual puppet run (as he asks at the end of the email).
Precisely again.
> My first thought is that the only accurate way to do this is by doing
> a full puppet run, as any part of the node's manifest could effect
> the variables used in the zone file template. You should be able to
> get away with a --noop run so the changes won't actually be applied
> (because noop does generate files from templates, but you would need
> to have a way to capture the newly generated zone file on the client
> host and run the named-checkzone there.
That's what I figured. I was hoping that there would be an easier way
that could somehow do a facter run but only run the minimal amount
of puppet.
Jon
Aaron Grewell
Perhaps just add an exec step or two to your regular run? Put the file in a temp location then move it to the live one if it checks out?
Dan White
Brian Gupta
I suggest this because bind should ignore bad zone files, and keep serving the old zones, when reloaded via rndc.
Maybe the answer is to also incorporate named-checkzone and some sort of email notification into the restart section of your init script.
As always test, and you milage may vary.
Roger
letting puppet do a:
/sbin/service named stop and then start
I have it do
/sbin/service named reload.
Which on redhat only runs
/usr/sbin/rndc reload
If that fails, named keeps running with the previous config it loaded
with no interruption, puppet complains that the restart failed, I get
notified and fix the zone file.
On Sep 11, 12:00 am, Aaron Grewell <aaron.grew...@gmail.com> wrote:
> Perhaps just add an exec step or two to youir regular run? Put the file in a
Nigel Kersten
On 9/10/2011 6:57 PM, Jonathan Stanton wrote:Precisely. Maybe later I'll face the issue of ruby syntax
Maybe I'm missing something here, but I think Jon was asking
something a bit different -- he doesn't want to check the validity of
the erb template (i.e. ruby syntax check) but syntax check the named
zone file generated by the template.
problems but right now I need to detect named syntax errors
before they cause problems.
Dominik Zyla
named-checkzone(8) should be suitable tool for your needs.
Best,
--
Dominik Zyla
Jon Forrest
> named-checkzone(8) should be suitable tool for your needs.
I already knew that. To repeat, what I'm trying to find out
is a way to get an expanded template without doing a full
run of puppet. I could then run named-checkzone on that
and then do a regular run.
In fact, this is only a special case of the more
general problem of getting access to any file generated
by a puppet run without actually putting the files
in the ultimate destination. In other words, it would
be like a --test run except files will be generated and
kept.
Jon Forrest
Russell Howe
> In fact, this is only a special case of the more
> general problem of getting access to any file generated
> by a puppet run without actually putting the files
> in the ultimate destination. In other words, it would
> be like a --test run except files will be generated and
> kept.
I have two ideas. The first is to introduce a preflight check run stage
which has a clone of whatever you use to generate the zonefile, but puts
it somewhere else and then does an exec with named-checkzone etc. If run
stage "main" depends on "preflight" that should do the right thing
assuming that failures of earlier run stages prevent execution of later
ones.
I'm not sure if that could lead to a race condition should someone
update the data between the preflight check and the main manifest run
though. Also you may find that your preflight stage requires things
which are in your main stage (installation of packages etc) which will
just get messy.
The other option which I think I prefer is to have puppet deploy your
zone files to a staging area, exec named-checkzone and whatever else you
want to do and then have an exec or similar which moves the staged
zonefiles into place on your nameserver. Make that exec depend on your
sanity checks and it will only run if the zonefiles check out OK.
You could even go so far as to stage your zonefiles into a git repo or
similar on your nameserver and end up with something like:
vcsrepo { "/staging/location":
# Set up git repo
}
file { "/staging/location/zone.a":
.... # whatever gets zone.a there
}
exec { "check /staging/location/zone.a":
command => "named-checkzone /staging/location/zone.a",
require => [File["/staging/location/zone.a"],Vcsrepo["/staging/location"]],
}
exec { "commit staged zonefiles",
command => "git commit ...",
require => Exec["check /staging/location/zone.a"],
# ...
}
exec { "deploy zonefiles to nameserver",
command => "tar cp . -C /staging/location | tar xp -C
/var/lib/named/zones",
require => Exec["commit staged zonefiles"],
notify => Service["named"],
}
--
Russell Howe
rh...@moonfruit.com
Jon Jaroker
Here is a process that I follow to avoid the type of misconfiguration
problem you encountered. This process is used to *avoid* the mis-
configured system problem. It is not a direct test that Puppet will
generate a valid configuration file from a template.
1) A "local" environment is used to deploy the entire system (which is
called a 'role' for the system). While I may be developing a single
class, I still confirm that the entire role can be deployed to my
local environment. The local server is typically rebuilt 30-50 times
during the course of development of a new manifest. This allows me to
catch errors early, as many of the mistakes I make result in spurious
errors on Puppet.
The manifests are then committed to SVN and automatically deployed to
a DEV environment, which is identical to the production environment.
This allows live testing against my other servers. The manifests then
are merged to the 'trunk' branch and get deployed to a TESTING
environment where selenium regression tests exercise the applications
to make sure they still work as expected. The manifests get deployed
to QA for manual testing and review and finally to PRODUCTION.
2) Include configuration checks before making a configuration file
live.
One example is with the sudoers file. I use a file-fragment approach
to generate a sudoers testing file and only make that file live if the
check passes. Your zone file can use a similar check.
In situations where checks are not possible, I include a 'testing'
flag to prevent a configuration file from being made live. I manually
check the testing file to make sure it is valid. The APF firewall
script is an example where this is important. When I first deployed
APF, puppet exploded with spurious errors because I forgot to open a
port. I now confirm (using the below process) that the APF
configuration is valid before allowing APF to be started on the remote
servers.
3) Each of my servers has a Dokuwiki page that is generated by puppet
during the deployment.
I use a file fragments pattern for this as well, where each class and
define puts a dokuwik-formatted fragment into a specific directory.
The configuration templates are included into the dokuwiki file
fragments using this approach. These fragments are assembled into a
single wiki page that is SCP'd to my documentation server. I then
review the server's configuration directly on this wiki page to
confirm that the destination server is configured in the manner I
expect. In the case of APF, I then change the 'testing' flag to false
and re-run puppet to make the configuration live.
This is a process, not an answer to your original question about
Puppet generating the configuration file from a template without
actually applying the changes. That would be a nice feature which I
don't believe currently exists. Your mis-configured zone file would
have been spotted in steps 1 and 2; maybe step 3.
Regards,
Jon
Craig White
----
I use /etc/puppet/deployment_files for situations such as this.
Puppet pseudo code...
file {/etc/puppet/deployment_files/zone_name
source => puppet:///modules/bind/zone_name.erb
notify => Exec["zone_name.un-deploy"]
exec {"zone_name.un-deploy"
/bin/rm "/etc/puppet/deployment_files/zone_name.deployed"
exec {"zone_name.deploy"
touch "/etc/puppet/deployment_files/zone_name.deployed"
notify => Exec["test - deploy if test successful"]
exec {"test - deploy if test successful"
command => cat /etc/puppet/deployment_files/zone_name
unless => bind-checkzone /etc/puppet/deployment_files/zone_name > 0 AND ls ! /etc/puppet/deployment_files/zone_name.deployed
require => File["/etc/puppet/deployment_files/zone_name"]
notify => [ Class["bind::service"], Exec["zone_name.deploy"] ]
That's not all - you might want some notice statements but that's the general idea anyway.
Craig