Any ideas about this error with upgrading to 0.25.1?

19 views
Skip to first unread message

Larry Ludwig

unread,
Nov 4, 2009, 2:52:27 PM11/4/09
to Puppet Users
Hi I'm getting this error with a 0.25.1 puppetmaster and 0.25.1 node.

Nov 4 14:43:35 devcentos5 puppetd[26099]: (//network-config::base/File
[hosts]) Failed to retrieve current state of resource: Error 400 on
SERVER: Not authorized to call find on /file_metadata/network-config/
hosts/hosts.devcentos5 Could not retrieve file metadata for
puppet:///network-config/hosts/hosts.devcentos5: Error 400 on SERVER:
Not authorized to call find on /file_metadata/network-config/hosts/
hosts.devcentos5 at /home/puppet/development/modules/dist/network-
config/manifests/base.pp:73
Nov 4 14:43:35 devcentos5 puppetd[26099]: (//network-config::base/File
[resolv.conf]) Failed to retrieve current state of resource: Error 400
on SERVER: Not authorized to call find on /file_metadata/network-
config/hosts/resolv.devcentos5.conf Could not retrieve file metadata
for puppet:///network-config/hosts/resolv.devcentos5.conf: Error 400
on SERVER: Not authorized to call find on /file_metadata/network-
config/hosts/resolv.devcentos5.conf at /home/puppet/development/
modules/dist/network-config/manifests/base.pp:44
Nov 4 14:43:35 devcentos5 puppetd[26099]: (//network-config::base/File
[host.conf]) Failed to retrieve current state of resource: Error 400
on SERVER: Not authorized to call find on /file_metadata/network-
config/hosts/host.devcentos5.conf Could not retrieve file metadata for
puppet:///network-config/hosts/host.devcentos5.conf: Error 400 on
SERVER: Not authorized to call find on /file_metadata/network-config/
hosts/host.devcentos5.conf at /home/puppet/development/modules/dist/
network-config/manifests/base.pp:58
Nov 4 14:43:35 devcentos5 puppetd[26099]: (//network-config::base/File
[nsswitch.conf]) Failed to retrieve current state of resource: Error
400 on SERVER: Not authorized to call find on /file_metadata/network-
config/hosts/nsswitch.devcentos5.conf Could not retrieve file metadata
for puppet:///network-config/hosts/nsswitch.devcentos5.conf: Error 400
on SERVER: Not authorized to call find on /file_metadata/network-
config/hosts/nsswitch.devcentos5.conf at /home/puppet/development/
modules/dist/network-config/manifests/base.pp:29

while the server gives off the error:
Not authorized to call find on /file_metadata/network-config/hosts/
hosts.devcentos5
Not authorized to call find on /file_metadata/network-config/hosts/
resolv.devcentos5.conf
Not authorized to call find on /file_metadata/network-config/hosts/
host.devcentos5.conf

I'm using passenger and do not have an auth.conf file.. Any ideas?

Brandon Evans

unread,
Nov 4, 2009, 7:32:40 PM11/4/09
to puppet...@googlegroups.com
Larry Ludwig wrote:
> Hi I'm getting this error with a 0.25.1 puppetmaster and 0.25.1 node.
> I'm using passenger and do not have an auth.conf file.. Any ideas?

I'm pretty sure the auth.conf is required in puppet 0.25.x. Try adding
the default one to your puppet conf directory on the puppetmaster.

-brandon

Avi Miller

unread,
Nov 4, 2009, 7:34:29 PM11/4/09
to puppet...@googlegroups.com
Hi,

Brandon Evans wrote:
> I'm pretty sure the auth.conf is required in puppet 0.25.x. Try adding
> the default one to your puppet conf directory on the puppetmaster.

I'm happily running 0.25.1 without an auth.conf and haven't seen any issues.

cYa,
Avi

Brice Figureau

unread,
Nov 5, 2009, 4:52:49 AM11/5/09
to puppet...@googlegroups.com

auth.conf is not mandatory. If you don't have one, Puppet will
automatically create sane minimal rules (matching the default provided
auth.conf).

The error the OP posted comes from the file serving layer, so it's more
a fileserver.conf issue.
--
Brice Figureau
Follow the latest Puppet Community evolutions on www.planetpuppet.org!

Larry Ludwig

unread,
Nov 5, 2009, 7:07:45 AM11/5/09
to Puppet Users

> The error the OP posted comes from the file serving layer, so it's more
> a fileserver.conf issue.

Ok here's my fileserver.conf.
[plugins]
allow 127.0.0.1/32
allow 192.168.10.0/24
allow 192.168.11.0/24
[modules]
allow 127.0.0.1/32
allow 192.168.10.0/24
allow 192.168.11.0/24

Puppet traffic occurs over a private network. This config works with
0.24.8 clients, so why wouldn't it work with 0.25?

Larry Ludwig

unread,
Nov 6, 2009, 8:05:32 PM11/6/09
to Puppet Users
No one has a solution to my issue? The online docs and ticket system
mention nothing about this issue.

Then unfortunately I'll have to roll back to 0.24.8 since then it
appears 0.25 isn't ready for prime time yet.

--
Larry Ludwig
Empowering Media
1-866-792-0489 x600
Managed and Unmanaged Xen VPSes
http://www.hostcube.com/

James Turnbull

unread,
Nov 6, 2009, 11:10:03 PM11/6/09
to puppet...@googlegroups.com, Puppet Users
Larry

I have a similar configuration to you and run 0.25.1 and don't see
this issue.

What passenger version?

Regards

James Turnbull
http://www.james-turnbull.net

Larry Ludwig

unread,
Nov 7, 2009, 11:30:58 AM11/7/09
to Puppet Users
2.2.2 here is the list of:

*** LOCAL GEMS ***

actionmailer (2.0.2)
actionpack (2.0.2)
actionwebservice (1.2.6)
activerecord (2.0.2)
activeresource (2.0.2)
activesupport (2.0.2)
capistrano (2.5.3)
cgi_multipart_eof_fix (2.5.0)
daemons (1.0.10)
echoe (3.0.2)
fastthread (1.0.1)
gem2rpm (0.5.3)
gem_plugin (0.2.3)
haml (2.2.2)
highline (1.5.0)
hobofields (0.7.5)
hobosupport (0.8.5)
hoe (1.7.0, 1.5.3)
hpricot (0.8.1)
mislav-will_paginate (2.2.3)
mongrel (1.1.5)
mysql (2.7)
net-scp (1.0.1)
net-sftp (2.0.1)
net-ssh (2.0.8)
net-ssh-gateway (1.0.0)
passenger (2.2.2)
rails (2.0.2)
rake (0.8.3)
rip (0.0.5)
rubyforge (1.0.0)
RubyRRDtool (0.6.0)
sqlite3-ruby (1.2.5)
sys-proctable (0.7.6)

On Nov 6, 11:10 pm, James Turnbull <ja...@lovedthanlost.net> wrote:
> Larry
>
> I have a similar configuration to you and run 0.25.1 and don't see  
> this issue.
>
> What passenger version?
>
> Regards
>
> James Turnbullhttp://www.james-turnbull.net

Larry Ludwig

unread,
Nov 9, 2009, 3:39:45 PM11/9/09
to Puppet Users
Could it be related the files folder I have has sub directories??

drwxr-xr-x 5 puppet puppet 4096 Apr 21 2009 .
drwxr-x--- 4 puppet puppet 4096 Apr 7 2009 ..
lrwxrwxrwx 1 root root 6 Apr 7 2009 CentOS -> RedHat
drwxr-x--- 2 puppet puppet 4096 Apr 7 2009 Debian
drwxr-x--- 2 puppet puppet 4096 Jun 1 10:13 RedHat
lrwxrwxrwx 1 root root 6 Apr 7 2009 Ubuntu -> Debian
-rw-r--r-- 1 puppet puppet 269 Apr 7 2009 host.conf
drwxr-x--- 2 puppet puppet 4096 Apr 21 2009 hosts
-rw-r--r-- 1 puppet puppet 148 Apr 7 2009 hosts.conf
-rw-r--r-- 1 puppet puppet 216 Apr 21 2009 resolv.1.conf
-rw-r--r-- 1 puppet puppet 216 Apr 21 2009 resolv.2.conf
-rw-r--r-- 1 puppet puppet 216 Apr 21 2009 resolv.conf

Brice Figureau

unread,
Nov 9, 2009, 3:47:47 PM11/9/09
to puppet...@googlegroups.com
On 09/11/09 21:39, Larry Ludwig wrote:
>
> Could it be related the files folder I have has sub directories??

I don't think so.

IMHO, you should try to add some debug information to your master to see
why those requests are forbidden for you.

The code that checks if the file serving request is allowed is in:
lib/puppet/indirector/file_server.rb
Check the authorized? method.

Add there some Puppet.info or whatever to print request.node and
request.ip. Then compare this to your fileserver.conf.
If the entries don't match then you're fileserver.conf is not correct.

If the entry match, then the request is forbidden because it wasn't
possible to find the correct mount for the file request.

Hope that helps,
--
Brice Figureau
My Blog: http://www.masterzen.fr/

Larry Ludwig

unread,
Nov 10, 2009, 10:26:13 PM11/10/09
to Puppet Users
Nope not it.

Correct info:

Nov 10 22:22:38 archive puppetmasterd[19932]: Not authorized to call
find on /file_metadata/network-config/hosts/nsswitch.devcentos5.conf
request.node: devcentos5.empoweringmedia.net request.ip: 192.168.10.41

I modified indirection.rb to spew out this output.

Keep in mind this node works as a 0.24.8 node and does not once I
upgrade to 0.25.1

-L

Brice Figureau

unread,
Nov 11, 2009, 7:15:52 AM11/11/09
to puppet...@googlegroups.com
On 11/11/09 04:26, Larry Ludwig wrote:
>
> Nope not it.
>
> Correct info:
>
> Nov 10 22:22:38 archive puppetmasterd[19932]: Not authorized to call
> find on /file_metadata/network-config/hosts/nsswitch.devcentos5.conf
> request.node: devcentos5.empoweringmedia.net request.ip: 192.168.10.41
>
>
> I modified indirection.rb to spew out this output.
>
> Keep in mind this node works as a 0.24.8 node and does not once I
> upgrade to 0.25.1

The code is completely different between 0.24.8 and 0.25.1 which is why
you're seeing a difference.

Is network-config a module?
If yes, then your source url is not correct, all the sourced files (for
a module) should be in: <module>/files/<whatever>

Here the url is only <module>/<whatever>
That means puppet "thinks" it isn't a module but a regular mount. Since
your fileserver.conf doesn't contain the "network-config" mount, it
doesn't find any valid mount, so the request is forbidden.

What I suggest:
* move the files under network-config/files/

* open a redmine ticket so that we have a better error message, which
at least would give us the reason.

I don't think we can see this problem as a regression since 0.24,
because this wasn't supposed to work in 0.24.

Larry Ludwig

unread,
Nov 11, 2009, 9:52:18 AM11/11/09
to Puppet Users

> The code is completely different between 0.24.8 and 0.25.1 which is why
> you're seeing a difference.

I understand this, but what works with 0.24.8 should work with 0.25.1
no? Hence it's looking more like a bug.

>
> Is network-config a module?
> If yes, then your source url is not correct, all the sourced files (for
> a module) should be in: <module>/files/<whatever>
>

Yes.

But what about 0.24.8 clients that need access to it, the puppetmaster
output states change it to <module> once all 0.24.8 are gone and it's
a warning message so it should work as is no?

> Here the url is only <module>/<whatever>
> That means puppet "thinks" it isn't a module but a regular mount. Since
> your fileserver.conf doesn't contain the "network-config" mount, it
> doesn't find any valid mount, so the request is forbidden.
>
> What I suggest:
>   * move the files under network-config/files/

Do you mean the subfolders below files? I did a test for one file and
same result.

>
>   * open a redmine ticket so that we have a better error message, which
> at least would give us the reason.

Ok.

Brice Figureau

unread,
Nov 11, 2009, 10:30:38 AM11/11/09
to puppet...@googlegroups.com
Hi Larry,

I re-read the whole thread, and I might have been wrong.

Your module files are indeed placed in network-config/files, correct?
Based on the error message (which mentions network-config/hosts) I
thought you placed your files at the wrong place. My bad.

On 11/11/09 15:52, Larry Ludwig wrote:
>
>
>> The code is completely different between 0.24.8 and 0.25.1 which is why
>> you're seeing a difference.
>
> I understand this, but what works with 0.24.8 should work with 0.25.1
> no? Hence it's looking more like a bug.

Correct.

>> Is network-config a module?
>> If yes, then your source url is not correct, all the sourced files (for
>> a module) should be in:<module>/files/<whatever>
>>
>
> Yes.
>
> But what about 0.24.8 clients that need access to it, the puppetmaster
> output states change it to<module> once all 0.24.8 are gone and it's
> a warning message so it should work as is no?

We're not talking about the same thing. The warning is to make sure you
will prefix all your modules sourced file by the "modules" keyword to
let puppet know we're talking about the "modules" mount.

>> Here the url is only<module>/<whatever>
>> That means puppet "thinks" it isn't a module but a regular mount. Since
>> your fileserver.conf doesn't contain the "network-config" mount, it
>> doesn't find any valid mount, so the request is forbidden.
>>
>> What I suggest:
>> * move the files under network-config/files/
>
> Do you mean the subfolders below files? I did a test for one file and
> same result.

I meant nothing in fact. I misread your problem.

>>
>> * open a redmine ticket so that we have a better error message, which
>> at least would give us the reason.
>
> Ok.

Back to your issue: so we found that the module can't be found.
We have to find why.

Do you use environments?

The only reason I can see for not finding the module, is that module
doesn't exist in the environment the client think it is.

But I'm afraid we will never know if you don't add more debug info in
the various files that finds a module from its path.
The means adding more debug to:

Puppet::Indirector::FileServer#authorized?
Puppet::FileServing::Configuration#find_mount
...

Hope that helps,

Larry Ludwig

unread,
Nov 11, 2009, 10:47:31 AM11/11/09
to Puppet Users


On Nov 11, 10:30 am, Brice Figureau <brice-pup...@daysofwonder.com>
wrote:
> Hi Larry,
>
> I re-read the whole thread, and I might have been wrong.
>
> Your module files are indeed placed in network-config/files, correct?
> Based on the error message (which mentions network-config/hosts) I
> thought you placed your files at the wrong place. My bad.

Yup they are in the files folder.

> Do you use environments?
>
> The only reason I can see for not finding the module, is that module
> doesn't exist in the environment the client think it is.

Yes that is correct I use environments.

>
> But I'm afraid we will never know if you don't add more debug info in
> the various files that finds a module from its path.
> The means adding more debug to:
>
> Puppet::Indirector::FileServer#authorized?
> Puppet::FileServing::Configuration#find_mount

What info do you want?

Larry Ludwig

unread,
Nov 11, 2009, 10:49:19 AM11/11/09
to Puppet Users

Brice Figureau

unread,
Nov 11, 2009, 11:39:14 AM11/11/09
to puppet...@googlegroups.com
On 11/11/09 16:47, Larry Ludwig wrote:
>
>
>
> On Nov 11, 10:30 am, Brice Figureau<brice-pup...@daysofwonder.com>
> wrote:
>> Hi Larry,
>>
>> I re-read the whole thread, and I might have been wrong.
>>
>> Your module files are indeed placed in network-config/files, correct?
>> Based on the error message (which mentions network-config/hosts) I
>> thought you placed your files at the wrong place. My bad.
>
> Yup they are in the files folder.
>
>> Do you use environments?
>>
>> The only reason I can see for not finding the module, is that module
>> doesn't exist in the environment the client think it is.
>
> Yes that is correct I use environments.

Could it be that one of your environment doesn't have network-config?
Could it be that in 0.25 the client is in this environment instead of
being in the one you think it is in?
BTW, how do you tell the client to be in a particular environment?
There is currently a bug report about this (ie environment can only be
set on the client and not in external_nodes anymore).

>>
>> But I'm afraid we will never know if you don't add more debug info in
>> the various files that finds a module from its path.
>> The means adding more debug to:
>>
>> Puppet::Indirector::FileServer#authorized?
>> Puppet::FileServing::Configuration#find_mount
>
> What info do you want?

in Puppet::Indirector::FileServer#authorized?
you should print the value of mount and just after
configuration.split_path(request)

This will let us know if that's indeed the mount that can't be found.

If mount proves to be nil for this particular request, edit:
Puppet::FileServing::Configuration#find_mount

and print the "mount_name".

With that info, edit: Puppet::Module#path
change the code to:
environment.modulepath.collect { |path| File.join(path, name) }.find { |d|
Puppet.info("testing: %s -> exists?: %s" % [d, FileTest.exist?(d) ]
FileTest.exist?(d)
}
(sorry for the bad wrapping, thunderbird sucks at not word-wrapping)

Then check that for network-config it prints true...

Larry Ludwig

unread,
Nov 11, 2009, 12:14:50 PM11/11/09
to Puppet Users

Hi,

> Could it be that one of your environment doesn't have network-config?

Yes they do not. The 'development' env has newer code.

> Could it be that in 0.25 the client is in this environment instead of
> being in the one you think it is in?

Checked LDAP config and it is in fact in the correct environment.
Again the node works with 0.24.8.

> BTW, how do you tell the client to be in a particular environment?
> There is currently a bug report about this (ie environment can only be
> set on the client and not in external_nodes anymore).

via LDAP. then it still works with 0.24.8 nodes on a 0.25.1 client.

-L

Larry Ludwig

unread,
Nov 11, 2009, 12:23:36 PM11/11/09
to Puppet Users

> > BTW, how do you tell the client to be in a particular environment?
> > There is currently a bug report about this (ie environment can only be
> > set on the client and not in external_nodes anymore).

Which ticket # is it? Is it this one?
http://projects.reductivelabs.com/issues/2748

having the client set the environment you then will run into the
chicken and egg syndrome. ie the first time puppet on the node runs
it's assumes 'production' then you set the puppet.conf via some method
and then next round runs in the proper environment. So also flipping
between env will also have the same issue and be one run behind.

Without question it makes sense to have the puppetmaster determine
this, not the client.

Brice Figureau

unread,
Nov 11, 2009, 12:33:48 PM11/11/09
to puppet...@googlegroups.com
On 11/11/09 18:23, Larry Ludwig wrote:
>
>
>>> BTW, how do you tell the client to be in a particular environment?
>>> There is currently a bug report about this (ie environment can only be
>>> set on the client and not in external_nodes anymore).
>
> Which ticket # is it? Is it this one?
> http://projects.reductivelabs.com/issues/2748

Yes.

> having the client set the environment you then will run into the
> chicken and egg syndrome. ie the first time puppet on the node runs
> it's assumes 'production' then you set the puppet.conf via some method
> and then next round runs in the proper environment. So also flipping
> between env will also have the same issue and be one run behind.
>
> Without question it makes sense to have the puppetmaster determine
> this, not the client.

It was/is debatted on puppet-dev.
I think the consensus was that the master should be able to override the
client env, but read the whole thread for more details.

Brice Figureau

unread,
Nov 11, 2009, 12:38:52 PM11/11/09
to puppet...@googlegroups.com
On 11/11/09 18:14, Larry Ludwig wrote:
>
>
> Hi,
>
>> Could it be that one of your environment doesn't have network-config?
>
> Yes they do not. The 'development' env has newer code.

Then that's your issue, because of #2748.

>> Could it be that in 0.25 the client is in this environment instead of
>> being in the one you think it is in?
>
> Checked LDAP config and it is in fact in the correct environment.
> Again the node works with 0.24.8.

#2748 make that your 0.25.1 client is not in the environment that your
ldap says it must be in. So your client is in the environement which is
specified in puppet.conf.

>> BTW, how do you tell the client to be in a particular environment?
>> There is currently a bug report about this (ie environment can only be
>> set on the client and not in external_nodes anymore).
>
> via LDAP. then it still works with 0.24.8 nodes on a 0.25.1 client.

Because 0.24.8 clients don't send their environment to the master. So
the master look it up and comes with what your external_node system returns.

The way it works right now in 0.25 is that the client sends the
environments with every requests, so the master doesn't even try to look
at the external_nodes returned environment and happily do what the
client tells him to do.

Larry Ludwig

unread,
Nov 11, 2009, 2:49:31 PM11/11/09
to Puppet Users
Ugh, now I'm getting another error with Puppet. The above node now
works, but deploying it on another node I get this error:

"Could not retrieve catalog from remote server: Could not intern from
pson: Could not convert from pson: Could not find relationship target
''

I have set the environment variable in the puppet.conf

So I assume this is a completely different error.

-L

--
Larry Ludwig
Empowering Media
1-866-792-0489 x600
Managed and Unmanaged Xen VPSes
http://www.hostcube.com/

Peter Meier

unread,
Nov 11, 2009, 3:52:00 PM11/11/09
to puppet...@googlegroups.com
> Ugh, now I'm getting another error with Puppet. The above node now
> works, but deploying it on another node I get this error:
>
> "Could not retrieve catalog from remote server: Could not intern from
> pson: Could not convert from pson: Could not find relationship target
> ''

yes: http://projects.reductivelabs.com/issues/2770

cheers pete

Reply all
Reply to author
Forward
0 new messages