Configuring Puppet for OnApp Cloud

[ankush grover]

Hi Friends,

My company is soon to going to deploy a private cloud from OnApp in
the infrastructure. Task given to me is to install puppet agent when
any Cloud instance boots. After searching on the google found there
are 2 ways to do this:


* Create a template in which puppet agent is already installed and
configured to talk to Puppetmaster. The issue is the hostnames for
these Cloud instances are given by the user and puppet requires unique
hostnames.

* 2nd Option is run some scripts to install puppet agent, assign the
new hostname based on the ip and connect it to the Puppet Master and
on the Puppet master side accept the client without Admin
intervention.

I somebody could share his experience in configuring Puppet for Cloud
Instances. What is the best way to configure Puppet and also if
possible please share the configuration or how to that will be very
helpful.


Thanks & Regards

Ankush

[ankush grover]

Any update on this?

[Brian Gupta]

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

I'm not familiar with OnApp, but the typical pattern for bootstrapping
cloud instances is to pass them a templated shell script that
bootstraps puppet onto the machine.

Here is an example template/script which I use with Foreman
(http://www.theforeman.org/) ENC/provisioning system for bootstrapping
Ubuntu 12.04 EC2 nodes (Please feel free to follow up with any
questions.):

#! /bin/bash

echo "updating system time"
/usr/sbin/ntpdate -sub ntp.pool.org

echo "<%= @host %>" > /etc/hostname
hostname <%= @host %>

echo "PUT_A_DUBUG_PUBLIC_SSH_KEY_HERE" > /root/.ssh/authorized_keys

echo "Configuring apt"
cat > /etc/apt/sources.list << EOF
<%= snippets "precise-sources" -%>
EOF

apt-get update
apt-get -y install ruby ruby1.8 libshadow-ruby1.8 libruby1.8
wget http://production.cf.rubygems.org/rubygems/rubygems-1.8.24.tgz
tar xvzf rubygems-1.8.24.tgz
pushd rubygems-1.8.24
ruby setup.rb
popd
gem1.8 install -v 2.6.9 --no-rdoc --no-ri puppet
mkdir /etc/puppet
# and add the puppet and ruby-shadow package
#apt-get -y install puppet

echo "Configuring puppet"
cat > /etc/puppet/puppet.conf << EOF
<%= snippets "puppetbgllc.conf" -%>
EOF

/usr/bin/puppetd --config /etc/puppet/puppet.conf -o --tags
no_such_tag --no-daemonize

puppetd --verbose
exit 0

[ankush grover]

Thanks Brian.

My issue is with OnApp the end user can give his hostname while
configuring the details for the Cloud Instance. When the system boots
up how do I make puppetmaster accepts this as a client without manual
intervention from administrator and apply some default classes for
this host.

Unique hostname is must for Puppet and if the user changes the
hostname on his own then how the puppet client will talk to
Puppetmaster.

[Brian Gupta]

To be clear, unique hostnames are not a must. Unique certnames are,
which by default are based on hostnames, but they don't have to be.
You can programmatically generate those using something like UUID
(Which is what Foreman uses for cloud provisioning). See the following
for more info on UUIDs:
http://en.wikipedia.org/wiki/Universally_unique_identifier

-Brian

[Jeff McCune]

On Fri, Jun 1, 2012 at 1:39 AM, Brian Gupta <brian...@brandorr.com> wrote:

To be clear, unique hostnames are not a must. Unique certnames are,
which by default are based on hostnames, but they don't have to be.
You can programmatically generate those using something like UUID
(Which is what Foreman uses for cloud provisioning). See the following
for more info on UUIDs:
http://en.wikipedia.org/wiki/Universally_unique_identifier

Actually, it's the other way around.  Unique node names are a must, unique cert names are not.

You can use the same certificate for multiple nodes if you wish, though this configuration carries a higher security risk than unique cert names.

You can re-use the same cert name with something like this:

# puppet.conf

[agent]

certname = shared.cert

node_name_fact = fqdn

-Jeff

[ankush grover]

My approach will be like this

generate uuid through uuidgen command and put that in certname under puppet.conf
start the puppet client
on the server allow autosigning of the client machines and a default policy

The issue with this approach is if the puppet agent is not working
properly on a host it is difficult to know that exact host without
doing ssh onto the server and also, to apply different policies for a
particular host.

Using nodename as unique will be problem with Onapp cloud as the end
user will be setting the hostname and which might not be unique.

[Jeff McCune]

On Sun, Jun 3, 2012 at 11:48 PM, ankush grover <ankush...@gmail.com> wrote:
> My approach will be like this
>
> generate uuid through uuidgen command and put that in certname under puppet.conf
> start the puppet client
> on the server allow autosigning of the client machines and a default policy
>
> The issue with this approach is if the puppet agent is not working
> properly on a host it is difficult to know that exact host without
> doing ssh onto the server and also, to apply different policies for a
> particular host.
>
> Using nodename as unique will be problem with Onapp cloud as the end
> user will be setting the hostname and which might not be unique.

What you can do in this instance is set both certname and
node_name_value in puppet.conf. For the rest of this description
node_name_fact also works, the only difference is that the value is
pulled out of Facter instead of being a static string in puppet.conf.

The downside is that you need to map the certname to the nodename in
auth.conf on the master.

# Agent puppet.conf
[main]
certname = B72008C3-708C-460B-80F5-38C221F7A479
node_name_value = jeff.uuid

# Master auth.conf
# (Put this entry _above_ the existing entry for catalog requests
since Puppet stops searching auth rules when it finds the first match.

# Allow laptop nodes (UUID based dynamic hostnames, sort of like the cloud.
# This entry must come before the default catalog entry.
path ~ ^/catalog/([^/]+).uuid$
method find
allow B72008C3-708C-460B-80F5-38C221F7A479

# allow nodes to retrieve their own catalog (ie their configuration)
path ~ ^/catalog/([^/]+)$
method find
allow $1

[ankush grover]

With 100 of nodes this might not be practical (without manual
intervention) or not without doing ssh onto the master.

[Jeff McCune]

You're right, explicitly adding the UUID isn't practical for large
numbers of nodes. It's more suited to a monitoring system.

For arbitrary numbers of agents you could tweak the catalog
regular expression allow rule to match a portion of the cert name.

# puppet.conf on the agent
[main]
certname = jeff.uuid.ec7f5196-7f63-5f73-f18d-ca69afc5c24d
node_name_value = jeff.uuid

# auth.conf on the master (This requires Puppet 2.7.1 or later since
# it uses a regexp allow)

path ~ ^/catalog/([^/]+).uuid$
method find

allow /^$1\.uuid.*/

path ~ ^/catalog/([^/]+)$
method find
allow $1

Here's puppet generating a new key and getting a catalog on the first
run with autosign turned on:

root@pe-centos6:~/conf# puppet agent --confdir /tmp/jeff --test
info: Creating a new SSL key for jeff.uuid.ec7f5196-7f63-5f73-f18d-ca69afc5c24d
warning: peer certificate won't be verified in this SSL session
info: Caching certificate for ca
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
info: Creating a new SSL certificate request for
jeff.uuid.ec7f5196-7f63-5f73-f18d-ca69afc5c24d
info: Certificate Request fingerprint (md5):
E4:A9:CD:19:15:2F:EC:E0:4C:C7:16:85:E3:8C:00:12
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
info: Caching certificate for jeff.uuid.ec7f5196-7f63-5f73-f18d-ca69afc5c24d
info: Caching certificate_revocation_list for ca
info: Caching catalog for jeff.uuid
info: Applying configuration version '1338877114'
info: Creating state file /tmp/jeff/var/state/state.yaml
notice: Finished catalog run in 0.02 seconds


We have the ability to generate a unique certificate CN that works for
a single node and use it to get a catalog with a single run.

The two remaining hurdles are signing the certificate request and writing
the configuration file. We can insecurely work around the CSR issue
today with autosign. We're working to make this easier while
maintaining security with the sites project Daniel Sauble emailed the
list about recently.

The second problem is writing to the configuration file. What do you
think a puppet subcommand should look like that helps automate this?

Maybe something like:

puppet config write --section main \
certname=$(hostname).uuid.$(ruby -rubygems -e '\
require "guid"; puts Guid.new; \
')
puppet config write --section main \
node_name_value=$(hostname).uuid

-Jeff

[Kevin]

Have you considered an mcollective based puppet provisioner such as

https://github.com/ripienaar/mcollective-server-provisioner ( http://www.youtube.com/watch?v=lxeYYOhI0S0 )

This would handle the certificate creation and signing

and should expand to manage certname\hostname with ease. (though i would use RDNS or user-data 

(assuming onapp works similarily to AWS|openstack ) to set the hostname)

Kevin.

Ankush

[ankush grover]

Thanks Kevin and Jeff.

As of now I want to keep the implementation simple

* Use Hostname/ipaddress+uuid for certificates
* Enabled Autosigning on the Puppet Master


Is there any variable something like this(posted on the puppet list an hour ago)

export FACTER_FACT_AUTOSCALE_GROUP=webgroup-b

which can be used to pass to the puppet master what group this node
belongs to and on the puppet server side a policy can be applied based
on the group it belongs to.



Regards

Ankush

[jcbollinger]

On Jun 5, 3:37 am, ankush grover <ankushcen...@gmail.com> wrote:
> Thanks Kevin and Jeff.
>
> As of now I want to keep the implementation simple
>
> * Use Hostname/ipaddress+uuid for certificates
> * Enabled Autosigning on the Puppet Master
>
> Is there any variable something like this(posted on the puppet list an hour ago)
>
> export FACTER_FACT_AUTOSCALE_GROUP=webgroup-b
>
> which can be used to pass to the puppet master what group this node
> belongs to and on the puppet server side a policy can be applied based
> on the group it belongs to.

You can indeed add facts via variables in Facter's environment; in
this context that would mean the Puppet agent's environment. How you
would arrange for that depends on how you start the agent. For
example:

FACTER_autoscale_group=webgroup-b puppet agent --onetime --no-
daemonize

Puppet does not have a native concept of machine groups, but they can
be mapped pretty easily to classes. Somewhere in your manifests,
then, you would have code similar to this:

case ${autoscale_group} {
'webgroup-a': {
include 'autoscale::web_a'
}
'webgroup-b': {
include 'autoscale::web_b'
}
}


John