Puppet 3.0: Not authorized to call find on /file_metadata, more issues?

[Forrie]

I've seen mention of this error in several places, with different causes.  So before I posted here, I attempted to resolve this on my own.

I corrected the change from puppet:///files to puppet:/// in my manifests *.pp files.  

No changes were made to the auth.conf file, and I did note in the ChangeLog that:

Auth.conf differentiates between names and IPs – There’s a new allow_ip keyword in auth.conf if you want to permit IP addresses. (PR991)

But I see no mention of that on the docs page at http://docs.puppetlabs.com/guides/rest_auth_conf.html.  

Our auth.conf is simple, and basically has either "allow $1" or "allow *" both which appear to still be valid in 3.0.

Here's an example, a simple example, an ntp.conf file:

class ntp-client {
    file { "/etc/ntp.conf":
        owner   => root,
        group   => root,
        mode    => 644,
        source  => "puppet:///etc/ntp.conf",
        require => [ Package["ntp"] ],
                notify  => Service["ntpd"],
    }
    package { "ntp": 
                ensure => latest,
    }
    service { "ntpd":
        ensure     => running,
                hasrestart => true,
                subscribe  => File["/etc/ntp.conf"],
    }
} # ntp-client

The error I'm seeing in the puppet.log, on the client system:

Oct  1 20:02:28 test-fms puppet-agent[11062]: Starting Puppet client version 2.7.17
Oct  1 20:02:31 test-fms puppet-agent[11062]: (/Stage[main]/Ntp-client/File[/etc/ntp.conf]) Could not evaluate: Error 400 on SERVER: Not authorized to call find on /file_metadata/etc/ntp.conf Could not retrieve file metadata for puppet:///etc/ntp.conf: Error 400 on SERVER: Not authorized to call find on /file_metadata/etc/ntp.conf at /etc/puppet/manifests/classes/ntp-client.pp:10

 

The permissions from /etc/puppet/files are correct:

-rw-r--r--. 1 puppet puppet 446 Mar 31  2011 etc/ntp.conf

The client puppet.conf file doesn't have any custom references other than the basics.

[main]
server = ourpuppet.server.com
vardir = /var/lib/puppet
logdir = /var/log/puppet
rundir = /var/run/puppet
ssldir = $vardir/ssl
[agent]
classfile = $vardir/classes.txt
localconfig = $vardir/localconfig
syslogfacility = local4
report = true
listen = true

I ran puppet master in verbose mode and got these diagnostics:

Starting Puppet master version 3.0.0
Info: access[^/catalog/([^/]+)$]: allowing 'method' find
Info: access[^/catalog/([^/]+)$]: allowing $1 access
Info: access[/certificate_revocation_list/ca]: allowing 'method' find
Info: access[/certificate_revocation_list/ca]: allowing * access
Info: access[/report]: allowing 'method' save
Info: access[/report]: allowing * access
Info: access[/file]: allowing * access
Info: access[/certificate/ca]: adding authentication no
Info: access[/certificate/ca]: allowing 'method' find
Info: access[/certificate/ca]: allowing * access
Info: access[/certificate/]: adding authentication no
Info: access[/certificate/]: allowing 'method' find
Info: access[/certificate/]: allowing * access
Info: access[/certificate_request]: adding authentication no
Info: access[/certificate_request]: allowing 'method' find
Info: access[/certificate_request]: allowing 'method' save
Info: access[/certificate_request]: allowing * access
Info: access[/]: adding authentication any
Info: Inserting default '~ ^/node/([^/]+)$' (auth true) ACL
Info: Inserting default '/status' (auth true) ACL
Warning: Host is missing hostname and/or domain: one-host.ourdomain.com
Compiled catalog for one-host.ourdomain.com in environment production in 1.16 seconds
Info: mount[files]: allowing 10.101.0.0/24 access
Error: Error parsing fileserver configuration: wrong number of arguments (3 for 1); using old configuration
Error: Not authorized to call find on /file_metadata/etc/ntp.conf
Error: Not authorized to call find on /file_metadata/etc/sudoers
Error: Not authorized to call find on /file_metadata/files/etc/ssh/ssh_known_hosts
Error: Not authorized to call find on /file_metadata/files/etc/ssh/sshd_config
Error: Not authorized to call find on /file_metadata/etc/puppet/namespaceauth.conf
Error: Not authorized to call find on /file_metadata/etc/puppet/puppet.conf.agent
Error: Not authorized to call find on /file_metadata/etc/puppet/auth.conf
Error: Not authorized to call find on /file_metadata/etc/resolv.conf.test
     

I reviewed the docs at http://docs.puppetlabs.com/guides/file_serving.html and our config looks fine.

Reading through the issue at http://projects.puppetlabs.com/issues/16667, I'm not clear what the fix actually is.  But, our config has been unaltered.   We have unused modules in the /etc/puppet/modules directory, where most of the little stuff has been in /etc/puppet/manifests, referenced in site.pp by:

import "classes/*"
import "nodes.pp"

And it's worked thus far.

In the example above, with ntp-client, it's just a simple little ntp-client.pp file that references a file that should be transfered, nothing more.  So I don't see how or why that wouldn't work as-is.

The error above:

Error: Error parsing fileserver configuration: wrong number of arguments (3 for 1); using old configuration

Doesn't make any sense to me, as our config seems to be in line with the docs:

The fileserver.conf file is pretty simple:

[files]
path    /etc/puppet/files
allow   10.101.0.0/24
allow   10.103.0.0/24  

 In the log above, clearly the connection is authorized.

Info: mount[files]: allowing 10.101.0.0/24 access

I'm not sure of the reference to the error "Warning: Host is missing hostname and/or domain" as it's clearly a FQHN (yes, I edited it here).

So I'm pretty stumped here.   Our only other option is to just downgrade back to 2.7.x and wait for these issues to get worked out.

Any pointers would be appreciated.  

Thanks.

[Matthaus Owens]

In Puppet 3.x, allow directives are limited to hostnames, if you wish
to allow an ip address, the allow_ip directive should be used. This
was in response to CVE-2012-3408
(http://puppetlabs.com/security/cve/cve-2012-3408/).

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/RuqTerxKHxMJ.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.



--
Matthaus Owens
Release Manager, Puppet Labs

[Matthaus Owens]

Oh, sorry, you mention that you already found that changelog entry. I
misread that part.

[Forrie]

The ChangeLog and the PR are not clear about this.  In fact, the documentation is vague and doesn't really mention allow_ip at all.   This should be updated and made more clear?

I will give this a try later on, on a test system, and see if that solves the problem.

Thanks.

[Eric Sorenson]

Check out the example auth.conf that comes with the distribution, it's heavily commented and should point the way:

https://github.com/puppetlabs/puppet/blob/master/conf/auth.conf

[Jeff McCune]

On Tue, Oct 2, 2012 at 11:09 AM, Forrie <for...@gmail.com> wrote:
> The ChangeLog and the PR are not clear about this. In fact, the
> documentation is vague and doesn't really mention allow_ip at all. This
> should be updated and made more clear?

Forrie,

I agree this wasn't very clear. I too had a hard time finding the
information until Matthaus pointed me in the right direction. We're
currently working on updating the documentation at docs.puppetlabs.com
to be much more clear about the breaking changes in Telly that we're
aware of and we intend. I think this information is important because
it can be hard to tell the difference between a breaking change we
intended to be a breaking change and a breaking change in behavior
that is actually a bug.

The current list of change for the 3.0.0 release will always be
available at the following URL:
http://links.puppetlabs.com/telly_breaking_changes

If you're still having trouble figuring out if a change in behavior is
intentional or is a bug, and the information at the above URL isn't
helpful, then please don't hesitate to ping me on IRC. I'll be
hanging out in #puppet-dev all week and my #1 priority this week is
working with the community on 3.0.0 related issues. My handle is
jmccune.

I hope this helps,
-Jeff

[Forrie]

Jeff, 

Thanks for the reply.  This is the first Puppet distribution that I've upgraded to that required a lot of manual changes.  But that comes with the territory :-)

What I think would be very useful is to not only include sample *.conf files, init scripts, and such, but also example usage of new features in common scenarios.  For example, Eric S. referred to the config file https://github.com/puppetlabs/puppet/blob/master/conf/auth.conf which does state allow_ip in the top portion, but there's no usage example in the content.   I sometimes find it easier to grok changes when I see contextual examples :-)   That might be a bad example, as it's pretty simple -- but I think you get the gist of what I mean.

Thanks again!

Forrest

[Forrie]

I've just built a staging system, to work out the issues I've been having with Puppet 3.x.  We now have 3.0.1 installed there.

I am again running into this fileserver issue and the same errors.   I read through some complaints here and I see mention that auth.conf is only able to use allow_ip.  In that file, I have "allow *" under "path /file" which should allow everyone.   I read that the allow_ip is not yet working for fileserver.conf.  My fileserver.conf has "allow 192.168.0.0/24" which was working until the upgrade.

So, can someone explain to me in plain english how we're supposed to get this working properly now.   I read through more notes and I don't see mention of this in upgrading, etc.   Perhaps I missed something -- I just want to get it working.  If it's an outstanding but that is preventing Puppet from working right now, I would think this would be a high-priority fix :-)

Thanks.

[Alex C]

I ran into a similar issue when upgrading to 3.0.  Make sure your specify 'modules' when loading puppet:// resources that are includes in modules:

source  => "puppet:///modules/etc/ntp.conf",

[Forrie]

Still no go.  I have a bunch of simple class files stored in /etc/puppet/manifests/classes -- not sure how that got designed, but I followed a book or two.  It's been working so far, until 3.0.   Here is the error when I prepend "modules" to the statement:

Oct 19 14:41:47 stage1 puppet-agent[21124]: Ignoring --listen on onetime run

Oct 19 14:41:48 stage1 puppet-master[21126]: Starting Puppet master version 3.0.1

Oct 19 14:41:48 stage1 puppet-master[21155]: Denying access: Forbidden request: stage1.myserver.com(127.0.0.1) access to /node/stage1.myserver.com [find] authenticated  at :95

Oct 19 14:41:48 stage1 puppet-master[21155]: Forbidden request: stage1.myserver.comu(127.0.0.1) access to /node/stage1.de-prod.harvard.edu [find] authenticated  at :95

Oct 19 14:41:48 stage1 puppet-agent[21124]: Unable to fetch my node definition, but the agent run will continue:

Oct 19 14:41:48 stage1 puppet-agent[21124]: Error 403 on SERVER: Forbidden request: stage1.myserver.com(127.0.0.1) access to /node/stage1.myserver.com [find] authenticated  at :95

Oct 19 14:41:48 stage1 puppet-master[21155]: Error parsing fileserver configuration: wrong number of arguments (3 for 1); using old configuration

Oct 19 14:41:51 stage1 puppet-master[21155]: Compiled catalog for stage1.myserver.com in environment production in 1.79 seconds

Oct 19 14:41:54 stage1 puppet-agent[21124]: (/Stage[main]/Ntp-client/File[/etc/ntp.conf]) Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/etc/ntp.conf

Oct 19 14:41:54 stage1 puppet-agent[21124]: (/Stage[main]/Ntp-client/Service[ntpd]) Dependency File[/etc/ntp.conf] has failures: true

Oct 19 14:41:54 stage1 puppet-agent[21124]: (/Stage[main]/Ntp-client/Service[ntpd]) Skipping because of failed dependencies

Oct 19 14:41:54 stage1 puppet-agent[21124]: Finished catalog run in 1.85 seconds

I figured "modules" wouldn't work as it's not there anyway.   So, moving it back to the normal URL, which would refer to the file as puppet:///etc/ntp.conf (removing "files" as the docs say), I get this:

Oct 19 14:45:17 stage1 puppet-agent[21635]: Ignoring --listen on onetime run

Oct 19 14:45:18 stage1 puppet-master[21638]: Starting Puppet master version 3.0.1

Oct 19 14:45:18 stage1 puppet-master[21667]: Denying access: Forbidden request: stage1.myserver.com(127.0.0.1) access to /node/stage1.myserver.com [find] authenticated  at :95

Oct 19 14:45:18 stage1 puppet-master[21667]: Forbidden request: stage1.myserver.com(127.0.0.1) access to /node/stage1.myserver.com [find] authenticated  at :95

Oct 19 14:45:18 stage1 puppet-agent[21635]: Unable to fetch my node definition, but the agent run will continue:

Oct 19 14:45:18 stage1 puppet-agent[21635]: Error 403 on SERVER: Forbidden request: stage1.myserver.com(127.0.0.1) access to /node/stage1.de-prod.harvard.edu [find] authenticated  at :95

Oct 19 14:45:18 stage1 puppet-master[21667]: Error parsing fileserver configuration: wrong number of arguments (3 for 1); using old configuration

Oct 19 14:45:19 stage1 puppet-master[21667]: Compiled catalog for stage1.myserver.com in environment production in 0.69 seconds

Oct 19 14:45:22 stage1 puppet-master[21667]: Not authorized to call find on /file_metadata/etc/ntp.conf

Oct 19 14:45:22 stage1 puppet-agent[21635]: (/Stage[main]/Ntp-client/File[/etc/ntp.conf]) Could not evaluate: Error 400 on SERVER: Not authorized to call find on /file_metadata/etc/ntp.conf Could not retrieve file metadata for puppet:///etc/ntp.conf: Error 400 on SERVER: Not authorized to call find on /file_metadata/etc/ntp.conf

Oct 19 14:45:22 stage1 puppet-agent[21635]: (/Stage[main]/Ntp-client/Service[ntpd]) Dependency File[/etc/ntp.conf] has failures: true

Oct 19 14:45:22 stage1 puppet-agent[21635]: (/Stage[main]/Ntp-client/Service[ntpd]) Skipping because of failed dependencies

Oct 19 14:45:22 stage1 puppet-agent[21635]: Finished catalog run in 1.98 seconds

So I'm back to square one.  From what I've read, there is an error with auth_ip not being recognized in fileserver.conf -- so, fine, why doesn't the older behavior not work?   The auth.conf file just has * next to the allow, which should still work.

The error message makes no sense to me:

Error parsing fileserver configuration: wrong number of arguments (3 for 1)

There is only one arg after "allow" - it doesn't seem to care what I put in there at this point.

I'm baffled (and getting really frustrated with this).

Thanks.

[Forrie]

To make matters worse, I'm referring to the documentation on puppetlabs.com that seems outdated for this newer release.

So I try all of these combinations and they do not work in fileserver.conf:

allow *

allow *.subdomain.company.com

allow xx.xx.xx.xx (Ip address)

Am I missing something here?

My auth.conf hasn't changed, it has simple * that should work fine.

[Jakov Sosic]

I would be interested too, because I've hit the same problem with both
3.0.0 and 3.0.1:

This is my /etc/puppet/fileserver.conf:

[files]
path /etc/puppet/files
deny *
allow *

And client says:

Could not evaluate: Error 400 on SERVER: Not authorized to call find on

/file_metadata/files/users/home/user/jsosic Could not retrieve file
metadata for puppet:///files/users/home/user/jsosic: Error 400 on

SERVER: Not authorized to call find on

/file_metadata/files/users/home/user/jsosic

I've tried:

chown -R puppet:puppet /etc/puppet/files

just in case, but no luck.


Solution I found so far is to remove "deny *" and leave only "allow *".
"allow 192.168.0.0/16" doesn't work too. So this is definitely broken in
many ways.... Documentation clearly says "allow takes precedence".

Anyway, this is just testing installation so I'll leave it as is... My
fileserver.conf looks like this now:

[files]
path /etc/puppet/files
allow *

and that works...



--
Jakov Sosic
www.srce.unizg.hr

[Nick Fagerlund]

Hi everybody,

This is a bug, and we're working on it. I'm about to update the auth.conf docs with info about allow_ip. Additionally, have a complete workaround for anyone being affected by this today. This offers complete equivalence to the fileserver.conf behavior that worked in 2.x and broke in 3.0.0.

**In fileserver.conf:**

Put the name of your mount point, the path, and an `allow *` directive.

    [files]
      path /etc/puppet/files
      allow *

**In auth.conf:**

Use a regular expression path to match both the `file_metadata` and `file_content` endpoints followed by the name of your custom mount point. Then, use any combination of `allow` and `allow_ip` directives to control access.

    path ~ ^/file_(metadata|content)/files/
    auth yes
    allow /^(.+\.)?example.com$/
    allow_ip 192.168.100.0/24

**Effect:**

This fully re-implements the previous behavior of the following fileserver.conf:

    [files]
      path /etc/puppet/files
      allow *.example.com
      allow 192.168.100.0/24

It does this by:

* Allowing any request to fulfill fileserver.conf's requirements...
* ...but using auth.conf to ensure that only authorized requests ever reach fileserver.conf.

This pattern will be forward-compatible with whatever permanent fix we implement for this bug. It's not very attractive, but it works.

[Forrie]

Wow, thank you Nick for taking the time to post that.  Much appreciated!

I have moved Puppet 3.0.x onto a staging system to test out all the kinks before I upgrade our primary master.   So I will test out your suggestions -- otherwise this bug pretty much brings all of our Puppet down and makes it useless.

Thank you.

[Jeff McCune]

On Wed, Oct 24, 2012 at 1:28 AM, imd <ivan.deg...@gmail.com> wrote:

Changed the auth.conf and fileserver.conf in a way you suggested above, now client gives another error:

err: /Stage[main]/Profiles/File[/etc/profile.d/wrkdir.py]: Could not evaluate: Error 400 on SERVER: Not authorized to call find on /file_metadata/files/shell/wrkdir.py Could not retrieve file metadata for puppet:///files/shell/wrkdir.py: Error 400 on SERVER: Not authorized to call find on /file_metadata/files/shell/wrkdir.py at /etc/puppet/manifests/classes/profiles.pp:42

With '/files/' or without, same error.

I'm taking a look at reproducing this issue and augmenting the work-around.  I hope to follow up with more information shortly.  My suspicion is that there isn't an auth rule that matches the file_metadata call, which is a different REST URI than the file content call.

-Jeff

[Jeff McCune]

On Wed, Oct 24, 2012 at 1:28 AM, imd <ivan.deg...@gmail.com> wrote:

Changed the auth.conf and fileserver.conf in a way you suggested above, now client gives another error:

err: /Stage[main]/Profiles/File[/etc/profile.d/wrkdir.py]: Could not evaluate: Error 400 on SERVER: Not authorized to call find on /file_metadata/files/shell/wrkdir.py Could not retrieve file metadata for puppet:///files/shell/wrkdir.py: Error 400 on SERVER: Not authorized to call find on /file_metadata/files/shell/wrkdir.py at /etc/puppet/manifests/classes/profiles.pp:42 

With '/files/' or without, same error.

Did you add the auth.conf rules before the default `path /file` rule.

For example, please see https://gist.github.com/3947951

Note how the default auth.conf file allows all requests.  The rules Nick mentioned need to go before the path /file rule on line 74, otherwise they'll never be reached because "first match wins."

I was able to fully work-around the issue by using these rules, also shown in the patch files included in the gist:

# JJM Lock down the "files" fileserver mount exported from filserver.conf

# Remember, this file is parsed top to bottom and the first match "wins" so

# more specific rules need to be above more generalized rules.

# The following two rules mean the agent must posses a signed certificate and

# must be connecting from the 192.168.0.0/16 subnet.

path /file_metadata/files

auth yes

allow_ip 192.168.0.0/16

path /file_content/files

auth yes

allow_ip 192.168.0.0/16

# unconditionally allow access to all file services

# which means in practice that fileserver.conf will

# still be used

path /file

allow *

Please note, I think Nick's original suggestion is slightly incorrect because it should now contain the "allow *.example.com" statement, as this would allow all agents who poses a signed certificate with a CN ending in example.com, regardless of their IP address.

If you truly want to authorize based on IP address, please stick with `auth yes`, which means agents must posses a signed, trusted certificate, and allow_ip which will grant access.

-Jeff

[Nick Fagerlund]

On Wednesday, October 24, 2012 11:39:50 AM UTC-7, Jeff McCune wrote:

Please note, I think Nick's original suggestion is slightly incorrect because it should now contain the "allow *.example.com" statement, as this would allow all agents who poses a signed certificate with a CN ending in example.com, regardless of their IP address.

Hmm,  really? I thought shell-style globbing didn't work in auth.conf allow directives, or at least that's what I discovered way back in the day. When we added globbing in 2.7.1, we implemented it with regular expressions instead of shell-style globs (http://docs.puppetlabs.com/guides/rest_auth_conf.html#allow), hence the  allow /^(.+\.)?example.com$/ line in my example.

[Jeff McCune]

Right, sorry.  The rules you posted are OK, it just took me a minute to grok them.

-Jeff

[Forrie]

I applied the fixes to my test/staging config and it's not very happy.   I think I'll just wait for the official fix to be out before I move forward with 3.x. :-)

For giggles, here's the log:

# puppet agent --test

Ignoring --listen on onetime run

Warning: Unable to fetch my node definition, but the agent run will continue:

Warning: Error 403 on SERVER: Forbidden request: stage1.myserver.com(127.0.0.1) access to /node/stage1.myserver.com [find] authenticated  at :102

Info: Retrieving plugin

Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate: Error 403 on SERVER: 

Forbidden request: stage1.myserver.com(127.0.0.1) access to /file_metadata/plugins [search] authenticated  at :102

Error: /File[/var/lib/puppet/lib]: Could not evaluate: Error 403 on SERVER: Forbidden request: stage1.myserver.com(127.0.0.1) access to /file_metadata/plugins [find] authenticated  at :102 Could not retrieve file metadata for puppet://stage1.myserver.com/plugins: Error 403 on SERVER: Forbidden request: stage1.myserver.com(127.0.0.1) access to /file_metadata/plugins [find] authenticated  at :102

Info: Caching catalog for stage1.myserver.com

Info: Applying configuration version '1351113815'

Error: /Stage[main]/Ntp-client/File[/etc/ntp.conf]: Could not evaluate: Error 403 on SERVER: Forbidden request: stage1.myserver.com(127.0.0.1) access to /file_metadata/etc/ntp.conf [find] authenticated  at :102 Could not retrieve file metadata for puppet:///etc/ntp.conf: Error 403 on SERVER: Forbidden request: stage1.myserver.com(127.0.0.1) access to /file_metadata/etc/ntp.conf [find] authenticated  at :102

/Stage[main]/Ntp-client/Service[ntpd]: Dependency File[/etc/ntp.conf] has failures: true

Warning: /Stage[main]/Ntp-client/Service[ntpd]: Skipping because of failed dependencies

Everything under /var/lib/puppet was created by the puppetmaster -- /var/lib/puppet/lib is owned by root:root as it is on my /working/ puppet master.  

[ fileserver.conf ]

[files]

path /etc/puppet/files

allow *

[ auth.conf ]

path ~ ^/file_(metadata|content)/files/

auth yes

allow /^(.+\.)?example.com$/

allow_ip 10.101.0.0/24

allow_ip 10.103.0.0/24

allow_ip 127.0.0.0/24

I tried the last one, 127/24, to see if the issue was with the client connecting locally; made no difference.

Everything else in auth.conf is "allow *"

We have a set of files in /etc/puppet/files/etc/blah-blah that are copied over to the clients.  They are not in a module (don't need to be).   I read somewhere that you need to put your files in the modules that belong to them, this doesn't apply here as far as I can tell.

In any case... that's all going off on a tangent.    I hope the fix will be out soon :-)

Thanks.

[Forrie]

No, I didn't leave *example.com* in my config - I put our own domain in there... just FYI ;-)

[Nick Fagerlund]

HMMMMMMM, this actually sounds like you've got a slightly larger problem, since can't get its own node object or its plugins. Any chance we could get a look at your whole auth.conf?

[Forrie]

auth.conf is below.

First, we have some simple classes that we use to manage files and packages that do not need to be in a module.   For example, /etc/puppet/files/etc/ntp.conf is a file I distribute to all our internal systems and I use this very simple recipe to manage them, which works fine under 2.7:

[ ntp-client.pp ]

class ntp-client {

    file { "/etc/ntp.conf":

        owner   => root,

        group   => root,

        mode    => 644,

        source  => "puppet:///etc/ntp.conf",

        require => [ Package["ntp"] ],

                notify  => Service["ntpd"],

    }

    package { "ntp":

                ensure => latest,

    }

    service { "ntpd":

        ensure     => running,

                hasrestart => true,

                subscribe  => File["/etc/ntp.conf"],

    }

} # ntp-client

From what I read in the docs, this /should/ work.  But it doesn't.   I shouldn't have to create a module path in order for this recipe to work (as I've seen suggested, or I've misunderstood).  

Here is the auth.conf file:

[ auth.conf ]

# This is an example auth.conf file, it mimics the puppetmasterd defaults

#

# The ACL are checked in order of appearance in this file.

#

# Supported syntax:

# This file supports two different syntax depending on how

# you want to express the ACL.

#

# Path syntax (the one used below):

# ---------------------------------

# path /path/to/resource

# [environment envlist]

# [method methodlist]

# [auth[enthicated] {yes|no|on|off|any}]

# allow [host|ip|*]

# deny [host|ip]

#

# The path is matched as a prefix. That is /file match at

# the same time /file_metadat and /file_content.

#

# Regex syntax:

# -------------

# This one is differenciated from the path one by a '~'

#

# path ~ regex

# [environment envlist]

# [method methodlist]

# [auth[enthicated] {yes|no|on|off|any}]

# allow [host|ip|*]

# deny [host|ip]

#

# The regex syntax is the same as ruby ones.

#

# Ex:

# path ~ .pp$

# will match every resource ending in .pp (manifests files for instance)

#

# path ~ ^/path/to/resource

# is essentially equivalent to path /path/to/resource

#

# environment:: restrict an ACL to a specific set of environments

# method:: restrict an ACL to a specific set of methods

# auth:: restrict an ACL to an authenticated or unauthenticated request

# the default when unspecified is to restrict the ACL to authenticated requests

# (ie exactly as if auth yes was present).

#

### Authenticated ACL - those applies only when the client

### has a valid certificate and is thus authenticated

# allow nodes to retrieve their own catalog (ie their configuration)

path ~ ^/catalog/([^/]+)$

method find

allow $1

# allow all nodes to access the certificates services

path /certificate_revocation_list/ca

method find

allow *

# allow all nodes to store their reports

path /report

method save

allow *

# inconditionnally allow access to all files services

# which means in practice that fileserver.conf will

# still be used

# path /file

# allow *

# allow_ip 10.101.0.0/24

# allow_ip 10.103.0.0/24

# Note that nothing here works, regardless of the CIDR

path ~ ^/file_(metadata|content)/files/

auth yes

allow /^(.+\.)?example.com$/

allow_ip 10.0.0.0/8

### Unauthenticated ACL, for clients for which the current master doesn't

### have a valid certificate

# allow access to the master CA

path /certificate/ca

auth no

method find

allow *

path /certificate/

auth no

method find

allow *

path /certificate_request

auth no

method find, save

allow *

# this one is not stricly necessary, but it has the merit

# to show the default policy which is deny everything else

path /

# allow *

auth any

 

Here are some of the errors I'm seeing today.  I do not have any other modules or classes defined here, just the ntp-client.pp on the staging system:

Oct 30 17:50:38 stage1 puppet-agent[3421]: catalog supports formats: b64_zlib_yaml dot pson raw yaml; using pson
Oct 30 17:50:38 stage1 puppet-agent[3421]: Caching catalog for stage1.mydomain.com
Oct 30 17:50:38 stage1 puppet-agent[3421]: Creating default schedules
Oct 30 17:50:38 stage1 puppet-agent[3421]: Loaded state in 0.00 seconds
Oct 30 17:50:38 stage1 puppet-agent[3421]: Applying configuration version '1351630198'
Oct 30 17:50:38 stage1 puppet-agent[3421]: (/Stage[main]/Ntp-client/Service[ntpd]/subscribe) subscribes to File[/etc/ntp.conf]
Oct 30 17:50:38 stage1 puppet-agent[3421]: (/Stage[main]/Ntp-client/File[/etc/ntp.conf]/require) requires Package[ntp]
Oct 30 17:50:38 stage1 puppet-agent[3421]: (/Stage[main]/Ntp-client/File[/etc/ntp.conf]/notify) subscribes to Service[ntpd]
Oct 30 17:50:38 stage1 puppet-agent[3421]: (/Schedule[daily]) Skipping device resources because running on a host
Oct 30 17:50:38 stage1 puppet-agent[3421]: (/Schedule[monthly]) Skipping device resources because running on a host
Oct 30 17:50:38 stage1 puppet-agent[3421]: (/Schedule[hourly]) Skipping device resources because running on a host
Oct 30 17:50:38 stage1 puppet-agent[3421]: Prefetching yum resources for package
Oct 30 17:50:38 stage1 puppet-agent[3421]: Executing '/bin/rpm --version'
Oct 30 17:50:38 stage1 puppet-agent[3421]: Executing '/bin/rpm -qa --nosignature --nodigest --qf '%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}
Oct 30 17:50:38 stage1 puppet-agent[3421]: ''
Oct 30 17:50:38 stage1 puppet-agent[3421]: Executing '/usr/bin/python /usr/local/lib/ruby/gems/1.8/gems/puppet-3.0.1/lib/puppet/provider/package/yumhelper.py'
Oct 30 17:50:40 stage1 puppet-agent[3421]: file_metadata supports formats: b64_zlib_yaml pson raw yaml; using pson
Oct 30 17:50:40 stage1 puppet-agent[3421]: (/Stage[main]/Ntp-client/File[/etc/ntp.conf]) Could not evaluate: Error 403 on SERVER: Forbidden request: stage1.mydomain.com(127.0.0.1) access to /file_metadata/etc/ntp.conf [find] authenticated  at :100 Could not retrieve file metadata for puppet:///etc/ntp.conf: Error 403 on SERVER: Forbidden request: stage1.mydomain.com(127.0.0.1) access to /file_metadata/etc/ntp.conf [find] authenticated  at :100
Oct 30 17:50:40 stage1 puppet-agent[3421]: (/Schedule[never]) Skipping device resources because running on a host
Oct 30 17:50:40 stage1 puppet-agent[3421]: (/Schedule[weekly]) Skipping device resources because running on a host
Oct 30 17:50:40 stage1 puppet-agent[3421]: (/Stage[main]/Ntp-client/Service[ntpd]) Dependency File[/etc/ntp.conf] has failures: true
Oct 30 17:50:40 stage1 puppet-agent[3421]: (/Stage[main]/Ntp-client/Service[ntpd]) Skipping because of failed dependencies
Oct 30 17:50:40 stage1 puppet-agent[3421]: (/Schedule[puppet]) Skipping device resources because running on a host
Oct 30 17:50:40 stage1 puppet-agent[3421]: Finishing transaction 23478903583320
Oct 30 17:50:40 stage1 puppet-agent[3421]: Storing state
Oct 30 17:50:40 stage1 puppet-agent[3421]: Stored state in 0.01 seconds
Oct 30 17:50:40 stage1 puppet-agent[3421]: Finished catalog run in 2.03 seconds
Oct 30 17:50:40 stage1 puppet-agent[3421]: Value of 'preferred_serialization_format' (pson) is invalid for report, using default (b64_zlib_yaml)
Oct 30 17:50:40 stage1 puppet-agent[3421]: report supports formats: b64_zlib_yaml raw yaml; using b64_zlib_yaml

[danielt]

Hi,

I am creating a folder with subfolders and files from within my basenode class to be existent on all servers with:

  file {"/sig/certstore/":

    source  => "puppet:///files/sig/certstore/",

    ensure => directory,

    recurse => true,

    owner   => tomcat6,

    group   => tomcat6,

    require => [Package['tomcat6'], File['/sig'],]

  }

}

My fileserver.conf looks like:

[files]

  

  path /etc/puppet/env/production/files

  allow *

[plugins] 

  allow *

My auth.conf looks like the work around from Nick:

### Authenticated ACL - those applies only when the client

### has a valid certificate and is thus authenticated

# allow nodes to retrieve their own catalog (ie their configuration)

path ~ ^/catalog/([^/]+)$

method find

allow $1

# allow nodes to retrieve their own node definition

path ~ ^/node/([^/]+)$

method find

allow $1

# allow all nodes to access the certificates services

path /certificate_revocation_list/ca

method find

allow *

# allow all nodes to store their reports

path /report

method save

allow *

path /file_metadata/files

auth yes

allow_ip 192.168.67.0/24

path /file_content/files

auth yes

allow_ip 192.168.67.0/24

# inconditionnally allow access to all files services

# which means in practice that fileserver.conf will

# still be used

path /file

allow *

### Unauthenticated ACL, for clients for which the current master doesn't

### have a valid certificate; we allow authenticated users, too, because

### there isn't a great harm in letting that request through.

# allow access to the master CA

path /certificate/ca

auth any

method find

allow *

path /certificate/

auth any

method find

allow *

path /certificate_request

auth any

method find, save

allow *

# this one is not stricly necessary, but it has the merit

# to show the default policy which is deny everything else

path /

auth any

Unfortunately I am still getting the error:

err: Could not run Puppet configuration client: Error 400 on SERVER: Not authorized to call find on /file_metadata/files/sig/certstore Could not retrieve file metadata for puppet:///files/sig/certstore: Error 400 on SERVER: Not authorized to call find on /file_metadata/files/sig/certstore at /etc/puppet/env/production/manifests/nodes.pp:64

I have been trying to play around but can't get it to work. It all worked like a charm on Puppet 2.7 but since my migration to 3.0.1 it's not working anymore.

[Nick Fagerlund]

Hi, Forrie,

I see a handful of problems here:

* First, you can't comment out the "path /file allow *" rule in auth.conf, because Puppet actually uses that for pluginsync and for files in modules! What you have to do is have BOTH the new rule I gave you AND the old one, but make sure the new one comes FIRST so that it can prevent the broader rule from being applied to the specific mount point you're trying to protect.

* Second, you copied my example "path ~ ^/file_(metadata|content)/files/" URL, but it looks like the actual mount point you're using is named "[etc]" in fileserver.conf. That means you should change the rule to "path ~ ^/file_(metadata|content)/etc/". You can see in the logs that the rule isn't matching the requests, both because the URL is wrong ( /file_metadata/etc/ntp.conf, whereas the rule is expecting  /file_metadata/files/ntp.conf) and because it's citing line 100 or 102 of auth.conf (meaning it skipped the rule).

* The IP address COULD be a problem, since you're connecting to the master from 127.0.0.1. Right now, it looks like you have a literal "example.com" in auth.conf and your real domain elsewhere, which would be a problem, but you probably just missed that when editing out your site-specific details. IF there's still an example.com in your auth.conf, and I realize there's probably not, you should replace it with your real domain.

Try fixing those and see where it gets you. Good luck!

[Nick Fagerlund]

Hey, Danielt,

I notice that you ONLY have allow_ip directives for that mountpoint. Is it possible that the agent you're running is connecting on a different network, or from the loopback address?

You might consider adding an " allow /^(.+\.)mydomain.com " directive to the rules, and see if the node can connect then.

[Forrie]

The files we have that go out are under /etc/puppet/files.  So, the ntp.conf exists under /etc/puppet/files/etc/ntp.conf -- there are also other structures under /etc/puppet/files that we refer to.   I would presume a more efficient way to refer to this is any file under /etc/puppet/files.     This is starting to get confusing :-)

Yeah, the example.com was a typo on my part.  I do have our own domain there.

Here's the relevant part of that config:

path ~ ^/file_(metadata|content)/files/etc/

path /file

auth yes

allow /^(.+\.)?ourdomain.com$/

allow_ip 10.0.0.0/8

which still generates errors:

# puppet agent --test

Ignoring --listen on onetime run

Warning: Unable to fetch my node definition, but the agent run will continue:

Warning: Error 403 on SERVER: Forbidden request: stage1.ourdomain.com(127.0.0.1) access to /node/stage1.ourdomain.com [find] authenticated  at :101

Info: Retrieving plugin

Info: Caching catalog for stage1.ourdomain.com

Info: Applying configuration version '1352241407'

Error: /Stage[main]/Ntp-client/File[/etc/ntp.conf]: Could not evaluate: Error 400 on SERVER: Not authorized to call find on /file_metadata/etc/ntp.conf Could not retrieve file metadata for puppet:///etc/ntp.conf: Error 400 on SERVER: Not authorized to call find on /file_metadata/etc/ntp.conf

/Stage[main]/Ntp-client/Service[ntpd]: Dependency File[/etc/ntp.conf] has failures: true

Warning: /Stage[main]/Ntp-client/Service[ntpd]: Skipping because of failed dependencies

Finished catalog run in 1.45 seconds

I think I found another bug.  I added the localhost with "allow_ip 127/8" which is a valid CIDR notation; I had to change it to "allow_ip 127.0.0.0/8" but it still failed anyway:

Warning: Unable to fetch my node definition, but the agent run will continue:

Warning: Error 403 on SERVER: Forbidden request: stage1.ourdomain.com(127.0.0.1) access to /node/stage1.ourdomain.com [find] authenticated  at :102

Do I also need to include "allow *" in the directive?  Seems that would take precedence over allow_ip.

Any word on when a fixed release is going to come out :-)

Thanks....

[Nick Fagerlund]

On Tuesday, November 6, 2012 2:44:25 PM UTC-8, Forrie wrote:

path ~ ^/file_(metadata|content)/files/etc/

path /file

auth yes

allow /^(.+\.)?ourdomain.com$/

allow_ip 10.0.0.0/8

This path is still wrong. The path in auth.conf needs to refer to the LOGICAL path that you set up with the mount point in fileserver.conf, not to the directory on disk.

In your puppet manifest we see the line:

        source  => "puppet:///etc/ntp.conf",

This means you must have a mount point called [etc] in fileserver.conf, right? So regardless of where that mount point points to on disk (/etc/puppet/files/etc), Puppet knows it as /etc. Therefore, the path in auth.conf MUST be

path ~ ^/file_(metadata|content)/etc/, NOT /files/etc/ or /files/. You cannot bulk-control everything in the /etc/puppet/files directory the way you're trying to, because puppet's fileserver does not know what /etc/puppet/files is. It knows about each mount point separately, and knows about them by the name you gave them in fileserver.conf. Put another way, your auth.conf rules must resemble the puppet:/// URLs you're using to access the files.

Fix that and you'll fix your problem.

Additionally, it looks like your auth.conf file doesn't have an entry for node objects yet, which is causing some unrelated errors. You can find out about that here:

http://docs.puppetlabs.com/puppet/3/reference/release_notes.html#puppet-agent-now-requires-node-access

Any word on when a fixed release is going to come out :-)

Nope, not yet.

[Forrie]

Ugh, thank you for taking the time to point all of that out.   It took me a little while, but I think it's working now in test mode.

Question, so for every directory underneath /etc/puppet/files, I need a separate stanza in fileserver.conf, so a new directory called /etc/puppet/files/blah would be in fileserver.conf

[blah]

path /etc/puppet/files/blah

allow *

AND in auth.conf

path ~ ^/file_(metadata|content)/blah/

auth yes

allow /^(.+\.)?harvard.edu$/

allow_ip 127.0.0.0/8

allow_ip 10.0.0.0/8

Then my [etc]

path ~ ^/file_(metadata|content)/etc/

auth yes

allow /^(.+\.)?harvard.edu$/

allow_ip 127.0.0.0/8

allow_ip 10.0.0.0/8

However, is /this/ necessary, too?

path ~ ^/file_(metadata|content)/

auth yes

allow /^(.+\.)?harvard.edu$/

allow_ip 127.0.0.0/8

allow_ip 10.0.0.0/8

Thanks!

[Nick Fagerlund]

On Friday, November 9, 2012 2:34:38 PM UTC-8, Forrie wrote:

However, is /this/ necessary, too?

path ~ ^/file_(metadata|content)/

auth yes

allow /^(.+\.)?harvard.edu$/

allow_ip 127.0.0.0/8

allow_ip 10.0.0.0/8

No. And it might interfere with some things. (For example, it would mean ONLY those machines could ever retrieve plugins or files from modules. Currently, the rule in auth.conf labeled "path file" allows all authenticated nodes to fetch plugins and module-served files.)

It sounds like you may want to read up again on how auth.conf works. http://docs.puppetlabs.com/guides/rest_auth_conf.html The salient part is that lookup proceeds linearly, and it stops at the first rule that matches the request. If that rule were above the other rules, no requests would ever reach them; the broad one on top would catch them all. If that rule were below the other rules, none of the requests relevant to the other rules would reach IT, since they'd catch them; it would only catch requests irrelevant to the other similar rules, such as pluginsync and module-served files.

Final addendum is that if you do want all of your mount points to have the exact same authorization rules, you can use a more complicated regex to do that, as long as it's matching the correct URLs. For example:

path ~ ^/file_(metadata|content)/(blah|etc|foo|bar|baz)/

[Felipe Salum]

Is this related to the same error I have when I run the puppet agent on my nodes ?

Nov 11 01:40:09 squeeze puppet-agent[8683]: Could not send report: Error 403 on SERVER: Forbidden request: puppetdb1.puppet.test(192.168.168.12) access to /report/puppetdb1.puppet.test [save] authenticated  at :67

I'm using Puppet 3.0.1

On Monday, October 1, 2012 5:48:25 PM UTC-7, Forrie wrote:

I've seen mention of this error in several places, with different causes.  So before I posted here, I attempted to resolve this on my own.

I corrected the change from puppet:///files to puppet:/// in my manifests *.pp files.  

No changes were made to the auth.conf file, and I did note in the ChangeLog that:

Auth.conf differentiates between names and IPs – There’s a new allow_ip keyword in auth.conf if you want to permit IP addresses. (PR991)

But I see no mention of that on the docs page at http://docs.puppetlabs.com/guides/rest_auth_conf.html.  

Our auth.conf is simple, and basically has either "allow $1" or "allow *" both which appear to still be valid in 3.0.

Here's an example, a simple example, an ntp.conf file:

class ntp-client {
    file { "/etc/ntp.conf":
        owner   => root,
        group   => root,
        mode    => 644,

        source  => "puppet:///etc/ntp.conf",

        require => [ Package["ntp"] ],
                notify  => Service["ntpd"],
    }
    package { "ntp": 
                ensure => latest,
    }
    service { "ntpd":
        ensure     => running,
                hasrestart => true,
                subscribe  => File["/etc/ntp.conf"],
    }
} # ntp-client

[Daniel Taschik]

I think I found the error for my problem. I had an allow_ip in my fileserver.conf which caused puppet to throw the following error in the master.log:

Puppet (err): Error parsing fileserver configuration: wrong number of arguments (3 for 1); using old configuration

Once I replaced it with allow * or allow *.domain.tld it worked. I can restrict the access to a certain IP(-range) or hostname in auth.conf but not in fileserver.conf.

Maybe this helps someone to fix their issue. 

[Nick Fagerlund]

On Saturday, November 10, 2012 5:43:48 PM UTC-8, Felipe Salum wrote:

Is this related to the same error I have when I run the puppet agent on my nodes ?

Nov 11 01:40:09 squeeze puppet-agent[8683]: Could not send report: Error 403 on SERVER: Forbidden request: puppetdb1.puppet.test(192.168.
168.12) access to /report/puppetdb1.puppet.test [save] authenticated  at :67

No, other than that they're both related to authentication in auth.conf. If you were upgrading from 2.6, note that the default value of the 'report' setting changed between 2.6 and 2.7:

http://docs.puppetlabs.com/references/2.7.latest/configuration.html#report
http://docs.puppetlabs.com/references/2.6.latest/configuration.html#report

So if your auth.conf file doesn't allow authenticated nodes to send save requests to /report, you will get errors. Examine your auth.conf file and compare it to the one here:

https://github.com/puppetlabs/puppet/blob/master/conf/auth.conf

You should have AT LEAST all the same rules, although your site may have some extra rules as well. Be aware that order matters in this file.

[Felipe Salum]

Hi Nick.

Actually this is a new environment I'm setting up using vagrant, puppet 3 and the default auth.conf.

I had to add allow_ip to the /reports request to make it work. Not sure why but it sometimes fail when using the puppet server provider from vagrant.

Thanks,
Felipe

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/rcFTBsu-IqkJ.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Felipe Salum]

I'm also having the same issue on the other locations. Not sure what's wrong since this is a default installation of puppet 3 with the original auth.conf

Error: /Stage[main]/Puppetdb::Master::Routes/File[/etc/puppet/routes.yaml]: Could not evaluate: Error 403 on SERVER: Forbidden request: puppet2.puppet.test(192.168.168.10) access to /file_metadata/modules/puppetdb/routes.yaml [find] at :102 Could not retrieve file metadata for puppet:///modules/puppetdb/routes.yaml: Error 403 on SERVER: Forbidden request: puppet2.puppet.test(192.168.168.10) access to /file_metadata/modules/puppetdb/routes.yaml [find] at :102

Error: Could not retrieve catalog from remote server: Error 403 on SERVER: Forbidden request: puppet2.puppet.test(192.168.168.10) access to /catalog/puppet2.puppet.test [find] at :101
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Error: Could not send report: Error 403 on SERVER: Forbidden request: puppet2.puppet.test(192.168.168.10) access to /report/puppet2.puppet.test [save] at :102


Maybe it is a naming resolution issue ? I'm using /etc/hosts since this is a vagrant environment only for testing purposes.

If I start updating auth.conf to use 'auth no' everywhere it passes.

I don't see the problem on my production servers, so it worries me more :)

To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.

[Felipe Salum]

I found out that I was missing a change in puppet.conf due using Puppetmaster Passenger:

from:
ssl_client_header = SSL_CLIENT_S_DN

to:
ssl_client_header = HTTP_X_SSL_SUBJECT

Now the permission issues are gone.

Weird enough that my Puppetmaster 2.7.x environment works without this change.

Felipe

[Forrie]

I see the ChangeLog in 3.0.2 and this bug is still not addressed?   Is there a technical problem that is not yet resolved, or is this just a matter of priority and time. 

[Eric Sorenson]

On Jan 3, 2013, at 2:02 PM, Forrie wrote:

> I see the ChangeLog in 3.0.2 and this bug is still not addressed? Is there a technical problem that is not yet resolved, or is this just a matter of priority and time.

Forrie this is on the table for 3.1 which will have a Release Candidate build Real Soon Now -- you can track progress on these two bugs:

https://projects.puppetlabs.com/issues/17448
https://projects.puppetlabs.com/issues/17449

Eric Sorenson - eric.s...@puppetlabs.com
#puppet irc: eric0

[Josh Cooper]

On Wed, May 22, 2013 at 12:39 AM, Russell Maclean <hyperthi...@gmail.com> wrote:

Thank you for clearing this up.
What I am not understanding is why this fails. If anyone could help me figure this out would be greatly appreciated.

[files]
path /etc/puppet/files
allow_ip 10.0.0.0/8

The allow_ip directive is not allowed in fileserver.conf, only auth.conf

 

But this works,

[files]
path /etc/puppet/files
allow *

See http://docs.puppetlabs.com/guides/file_serving.html#ip-addresses for more information.

 

It resulted in much hair loss.

On Tuesday, October 23, 2012 9:55:16 AM UTC+11, Nick Fagerlund wrote:

Hi everybody,

This is a bug, and we're working on it. I'm about to update the auth.conf docs with info about allow_ip. Additionally, have a complete workaround for anyone being affected by this today. This offers complete equivalence to the fileserver.conf behavior that worked in 2.x and broke in 3.0.0.

**In fileserver.conf:**

Put the name of your mount point, the path, and an `allow *` directive.

    [files]
      path /etc/puppet/files
      allow *

**In auth.conf:**

Use a regular expression path to match both the `file_metadata` and `file_content` endpoints followed by the name of your custom mount point. Then, use any combination of `allow` and `allow_ip` directives to control access.

    path ~ ^/file_(metadata|content)/files/
    auth yes
    allow /^(.+\.)?example.com$/

    allow_ip 192.168.100.0/24

**Effect:**

This fully re-implements the previous behavior of the following fileserver.conf:

    [files]
      path /etc/puppet/files
      allow *.example.com
      allow 192.168.100.0/24

It does this by:

* Allowing any request to fulfill fileserver.conf's requirements...
* ...but using auth.conf to ensure that only authorized requests ever reach fileserver.conf.

This pattern will be forward-compatible with whatever permanent fix we implement for this bug. It's not very attractive, but it works.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To post to this group, send email to puppet...@googlegroups.com.

Visit this group at http://groups.google.com/group/puppet-users?hl=en.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Josh Cooper

Developer, Puppet Labs

Join us at PuppetConf 2013, August 22-23 in San Francisco - http://bit.ly/pupconf13
Register now and take advantage of the Early Bird discount - save 25%!