extlookup inconsistencies

[Matt Robinson]

On Fri, Jul 23, 2010 at 3:25 PM, R.I.Pienaar <r...@devco.net> wrote:

There was some reports from joe-mac on irc that it's behaving inconsistently on 2.6, giving different results between different runs.

 

I'm tasked with helping to figure out these inconsistencies.  Does someone have some code the reproduces the problem?  Has anyone created a ticket?

I hacked at the code a bit to add a 4th option that will enable copious amount of debug messages to be printed to hopefully help sort this out.

Sounds great once we can reproduce the problem.  Is the code on github or somewhere public?

We should attempt to merge these I gues and ask someone who has 2.6 going to test, maybe Joe has some time?

I've got the time too once I know how to reproduce the problem, but the more testers the better.

Matt 

[Joe McDonagh]

I had some test output which clearly showed with no manifest changes, different values being applied on each run of puppet. Unfortunately the file contained somewhat sensitive data and I removed it from RIP's shell server.

I know that RI has seen the same thing I have; with some wrong resolv.conf files getting written out. The potential for a real mess is serious, I can attest that although I noticed fast, before I could stop puppet I lost access to a couple of boxes because of iptables rules getting changed to something that made no sense for where these boxes reside.

--
Joe McDonagh
Operations Engineer
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."

[Julian Simpson]

I know that RI has seen the same thing I have; with some wrong resolv.conf files getting written out. The potential for a real mess is serious, I can attest that although I noticed fast, before I could stop puppet I lost access to a couple of boxes because of iptables rules getting changed to something that made no sense for where these boxes reside.

I take it the extlookup tests ran against puppet 2.6 without problems?  I also left an 'acceptance' test on my adaption of extlookup that executed a trivial lookup via puppet.  Not sure if that survived and not really in a position to look right now.

J.
--

Julian Simpson

The Build Doctor Ltd.

http://www.build-doctor.com

jul...@build-doctor.com

(+44) 207 183 0323

[R.I.Pienaar]

hello,

----- "Matt Robinson" <ma...@puppetlabs.com> wrote:

It would help a lot if you didnt send html mail :)

I'll add the debug back to the current version of the code thats in 2.6.1rc1.

The behavior seems to be that once node a ran extlookup once, then node b will get the data from answers its resolved. As if somewhere some extra layer of caching was added that is now handing out the same answers.

The debugging I'll add produce something along these lines:

Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Doing extlookup search for syslocation
Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Adding nephilim.ml.org to precedence
Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Adding location_hetzner to precedence
Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Adding domain_ml.org to precedence
Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Adding country_de to precedence
Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Adding common to precedence
Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Reading file /etc/puppet/manifests/extdata/location_hetzner.csv
Jul 25 04:17:35 monitor3 puppetmasterd[703]: [syslocation] Found value Hetzner, Germany in /etc/puppet/manifests/extdata/location_hetzner.csv

This clearly shows the logic being followed and what lookupvar is reporting.

I didnt just want to emit these as Puppet.debug lines though since that will on heavy extlookup use be a bit overwhelming so I am leaning towards just adding a 4th optional argument. If given it will enable this kind of log at notice level, making it easy to just debug in isolation.

At this point its pretty hard for me to test this without a lot of work due to it being pretty destructive to any and all my machines, will see if I can get something going.

--
R.I.Pienaar

[R.I.Pienaar]

hello,

I've filed http://projects.puppetlabs.com/issues/4370 with my findings.

----- "Matt Robinson" <ma...@puppetlabs.com> wrote:

--
R.I.Pienaar