Debugging external node scripts

[Curt Micol]

Hello,
Alright, my puppetmaster configuration works without external nodes,
but I really need my external node script to work.

In my /etc/puppet/puppet.conf I have the following:
[main]
...
node_terminus = exec
external_nodes = /etc/puppet/tools/external_nodes.py

When running puppetmaster, I get:
[root@vpsadmins ~]# puppetmasterd --no-daemonize --verbose --debug
[--snip--]
info: Listening on port 8140
notice: Starting Puppet server version 0.24.8
debug: Overriding dc2-vps1-400.example.com with cert name dc2-
vps1-400b.example.com
info: access[fileserver]: allowing *.example.com access
info: access[puppetmaster]: allowing *.example.com access
info: access[resource]: allowing vpsadmins.example.com access
info: access[puppetbucket]: allowing *.example.com
access info: access
[puppetreports]: allowing *.example.com access
debug: Allowing authenticated client dc2-vps1-400b.example.com
(67.227.199.245) access to puppet
master.getconfig
debug: Our client is remote
debug: Executing '/etc/puppet/tools/external_nodes.py dc2-
vps1-400b.example.com'
err: Could not call: Could not find node 'dc2-vps1-400b.example.com';
cannot compile
notice: Caught INT; shutting down
debug: Signal caught here:
debug: /usr/lib/ruby/site_ruby/1.8/puppet/external/event-loop/event-
loop.rb:127:in `call'
debug: /usr/lib/ruby/site_ruby/1.8/puppet/external/event-loop/event-
loop.rb:127:in `select'
debug: /usr/lib/ruby/site_ruby/1.8/puppet/external/event-loop/event-
loop.rb:127:in `select'
debug: /usr/lib/ruby/site_ruby/1.8/puppet/external/event-loop/event-
loop.rb:116:in `iterate'
debug: /usr/lib/ruby/site_ruby/1.8/puppet/external/event-loop/event-
loop.rb:107:in `run'
debug: /usr/lib/ruby/site_ruby/1.8/puppet.rb:320:in `start'
debug: /usr/sbin/puppetmasterd:285
notice: Shutting down

When I run the script directly:
[root@vpsadmins ~]# python /etc/puppet/tools/external_nodes.py dc2-
vps1-400b.example.com
---
classes:
- custom
- monitoring::base
- ntpd
- puppetd
- rpms
- sshd
- yum
- crontab
- iptables
- ldap::client
- monitoring::vps
- sudo
- virtuozzo
- vpsscripts


Here's the script I am running:
#!/usr/local/bin/python
#
# external_nodes.py
# Take a YAML file containing node types, regular expressions to
match
# a hostname, and which modules are get loaded for each node type.
# Print to stdout the list of classes (in YAML) associated with the
node
# type. This script is currently not very pythonic.
#
# TODO: actual error checking, particularly for file handling

### Imports
import sys
import re
import time
import yaml

### Constants
# puppet user must have read acces to this
NODEFILE = '/etc/puppet/nodes.yaml'
# puppet user must have write access to this
LOGFILE = '/var/log/puppet/nodes.log'

### Arguments
hostname = sys.argv[1]

### Functions

# Open NODESFILE and load the two documents into structures, return as
tuple
def parse_nodefile():
f = file(NODEFILE, 'r')
docs = yaml.load_all(f.read())
f.close()
return (docs.next(), docs.next())

# Write a msg to LOGFILE
def log(msg):
f = file(LOGFILE, 'a')
timestamp = time.strftime('%Y%m%d-%H:%M')
f.write(timestamp + ' - ' + msg + '\n')
f.close()

### Action!

(regexes, modules) = parse_nodefile()

for nodetype, regexlist in regexes.iteritems():
for regex in regexlist:
p = re.compile(regex)
m = p.match(hostname)
if m:
found_nodetype = nodetype

modulelist = modules['default']

try:
if found_nodetype and modules[found_nodetype] is not None:
modulelist.extend(modules[found_nodetype])
except NameError:
log(hostname + ' doesn\'t match a defined node type')
sys.exit(1)

yamldoc = {'classes': modulelist}
print yaml.dump(yamldoc, explicit_start=True,
default_flow_style=False)

# Puppet expects a return code of 0 to signal to indicate success
# and non-zero for error or a non-regcognized hostname
sys.exit(0)

This particular server is covered by this in my nodes.yaml:
- ^dc2-vps[12]-[2-9][0-9][0-9]b\.example\.com$

According to the wiki all I need to do is return YAML with an exit
code of 0, which this script does.

Anyone have any ideas as to why I am getting this error?

I appreciate any help,

Curt Micol

[Nicolas Szalay]

Le lundi 29 juin 2009 à 01:09 -0700, Curt Micol a écrit :

> When I run the script directly:
> [root@vpsadmins ~]# python /etc/puppet/tools/external_nodes.py dc2-
> vps1-400b.example.com
> ---
> classes:
> - custom
> - monitoring::base
> - ntpd
> - puppetd
> - rpms
> - sshd
> - yum
> - crontab
> - iptables
> - ldap::client
> - monitoring::vps
> - sudo
> - virtuozzo
> - vpsscripts

Did you try running it as the "puppet" user ? permissions on NODEFILE &
LOGFILE could be source of errors.

Regards,

Nicolas

[Curt Micol]

2009/6/29 Nicolas Szalay <nsz...@qualigaz.com>:

> Le lundi 29 juin 2009 à 01:09 -0700, Curt Micol a écrit :
> Did you try running it as the "puppet" user ? permissions on NODEFILE &
> LOGFILE could be source of errors.

Yes, sorry, I get the same results with running as the puppet user.
Just tested once more to verify and received the same error.

--
# Curt Micol

[Martin Wheldon]

Hi Curt,

Did you manage to resolve this issue? If not I think it may be the
format of your yaml output.

>---
>classes:
>- custom
>- monitoring::base

Should be this

---
classes:
- custom
- monitoring::base

Note the indentation. Having said that I have been unable to persuade
the python yaml module to produce this
as yet.

Hope this helps

Martin

[Curt Micol]

On Fri, Jul 3, 2009 at 12:05 PM, Martin Wheldon<mwhe...@googlemail.com> wrote:
>
> Hi Curt,
>
> Did you manage to resolve this issue? If not I think it may be the
> format of your yaml output.
>
>>---
>>classes:
>>- custom
>>- monitoring::base
>
> Should be this
>
> ---
> classes:
>   - custom
>   - monitoring::base
>
> Note the indentation. Having said that I have been unable to persuade
> the python yaml module to produce this
> as yet.
>
> Hope this helps

Hey Martin,

Thanks for the response. I did indeed get this working, it turns out
it wasn't Puppet at all. The path for finding Python was incorrect, so
while I was pointing it at /usr/local/bin/python it wasn't finding it
and using /usr/bin/python which for some reason this script can't use
(version issues). How and why this is, I don't know, but that's about
as close to the reason as I was able to get.

I think YAML is white space agnostic due to the '-' and ':' syntax. I
could be wrong though.

I hope to get some time and send in a patch to update the output on
errors for executing external nodes, 4 days trying to figure out this
issue was quite annoying and the error did little to help remedy the
issue.

Thanks again for the reponse,

--
# Curt Micol