Moving from manifest files to ENC script - not working...

[shlo....@gmail.com]

Hi,

Until now I work with manifest file. Now I would like to create a perl/php script that will create the YAML script and should overwrite the manifest file.

So I edit the file puppet.conf and put:

mode_terminus = exec
external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=http://localhost:3000 /path/to/bin/external_node

In the file /path/to/bin/external_node I put my code for the perl script. when I run Puppet on the agent, it's look like it still take the manifest instead of the script I put.

How Can I make this script work instead of the manifest files? how can I debug it?

Also, How can I send parameters to the script? How to access to fact variables from the script, like $hostname etc?

Thanks.

[daddy dp]

As I know puppet merge both sources ENC and manifest file. You can use empty manifest file to avoid this behavior. Host name will be available as first argument to this script, all others facts you can query from puppetdb. Also you can use hiera as ENC, and all facts will be available for you out of the box.

[RichTea]

--
<-- http://23.me.uk/2 -->
<--Time flies like an arrow; fruit flies like a banana.  -->

On 16 June 2014 08:52, <shlo....@gmail.com> wrote:

Hi,

Until now I work with manifest file. Now I would like to create a perl/php script that will create the YAML script and should overwrite the manifest file.

So I edit the file puppet.conf and put:

mode_terminus = exec
external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=http://localhost:3000 /path/to/bin/external_node

​ dont think you need the extra option in your external_nodes... i use:

    node_terminus = exec
    external_nodes = /PATH/TO/ENC/findrole.rb

FYI My enc script is here, it reads json files though and can pass querys back of to the dashboard.
http://paste.debian.net/104832
​

 

In the file /path/to/bin/external_node I put my code for the perl script. when I run Puppet on the agent, it's look like it still take the manifest instead of the script I put.

How Can I make this script work instead of the manifest files? how can I debug it?

Also, How can I send parameters to the script? How to access to fact variables from the script, like $hostname etc?

Thanks.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/6eb837b7-b3d8-4531-ad1b-4f14c2826c60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[shlo....@gmail.com]

HI,

I empty the file nodes.pp, but it look like it's not running.

I make the script to print something to a file, each time it run, but it write it only when I run it from the command line,

While I expect to have that print each time the Puppet agent run the puppet command.

Is the hostname should send automatically as the first parameter or I need to do it in somehow?

When I install open source Puppet 3.3.1, I follow old instruction of 2.6.0.

I did not install the PuppetDB, unless it came by default installation. Is it installed with open source puppet 3.3.1?

Can ENC work without it or should I install PuppetDB? installation  can mess what I had?

Maybe I missing the basic,

I create perl script and put it in /path/to/bin/external_node

in this script I put a regular perl commands:

    #!/usr/bin/perl -w
    use strict;
    use YAML qw(Dump);
    
    my $hostname = shift || die "No hostname passed";
    
    system("echo $hostname >> /tmp/llkkll");
    
    my ($host, $domain, $net) = ($1, $2, $3);
    
    my @classes = ();
    my %parameters = (
        puppetserver => "puppetsrv"
    );
        @classes = ('class1', 'class2::basic', "class3");
        $parameters{var} = "kk1";
    
    
    print Dump({
        classes => \@classes,
        parameters => \%parameters,
    }); 

that as output give me:

---
classes:
  - class1
  - class2::basic
  - class3
parameters:
  puppetserver: puppetsrv
  var: kk1

Is it okay?

Thank you very much for your responses.

[Ken Barber]

> When I install open source Puppet 3.3.1, I follow old instruction of 2.6.0.
> I did not install the PuppetDB, unless it came by default installation. Is
> it installed with open source puppet 3.3.1?

No it is not. It's a separate step. Follow the documentation:
http://docs.puppetlabs.com/puppetdb/latest/

ken.

[shlo....@gmail.com]

Hi,

I understood  ENC can work without the PuppetDB installation. 

I cannot make ENC work and I cannot find a log or any way to debug it, so I can find the problem.

any idea are welcome.

Thanks.

[Jason Antman]

Yes, I have an idea.

1) Read the documentation: http://docs.puppetlabs.com/guides/external_nodes.html
2) If that doesn't help, post your ENC code, with specific examples of output and problems that you're having.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/85498348-93a0-49ff-bdc5-589f76e7ee79%40googlegroups.com.

[shlo....@gmail.com]

I read the documentation.

I send yesterday example of my perl ENC code  and the output.  I'm sending them again:

The definition of the classes are in manifests under the module directory.

My problem: 

when I run the puppet command on the agent, it look like it ignore the classes I define in ENC script

and only if I have the node definition in node.pp it try to so something.

Maybe I'm missing very basic issue. I'm sorry.

Is there any way to debug it? 

[Johan De Wit]

I think, in your yaml, you are missing the name: <certname>

Grts

jo

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5a15194a-c44e-4c3b-b62b-cc07e8094bf4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

-- 
Johan De Wit

Open Source Consultant

Red Hat Certified Engineer              (805008667232363)
Puppet Certified Professional 2013/2014 (PCP0000006)
_________________________________________________________
 
Open-Future                 Phone     +32 (0)2/255 70 70
Zavelstraat 72              Fax       +32 (0)2/255 70 71
3071 KORTENBERG             Mobile    +32 (0)474/42 40 73
BELGIUM                     http://www.open-future.be
_________________________________________________________
 

Upcoming Events:

Puppet Advanced Training | https://www.open-future.be/puppet-advanced-training-10-till-12th-june

Puppet Introduction Course | https://www.open-future.be/puppet-introduction-course-13th-june

Linux Training | https://www.open-future.be/linux-training-16-till-20th-june

Subscribe to our newsletter: http://eepurl.com/BUG8H

[jcbollinger]

Probably.  Hooking up an ENC is not very hard.

If the ENC is not running when a catalog request comes in then most likely the ENC it is not properly configured.  Supposing that you are running in agent / master mode, the ENC needs to be set up on the master.  There, the 'node_terminus' configuration parameter must be set to "exec" and the 'external_nodes' configuration parameter must be set to the absolute pathname of the ENC script.

Furthermore, the script must be accessible and executable by the master, which typically does not run as a privileged user.  Not only the script, but also all directories in the path to it must be readable and executable/traversable by the master.  File permissions and other access controls may come into play here.

Also, once it is properly configured, you need to restart the master in order for the new configuration to be noticed.

It is highly unlikely that the master is running the ENC successfully, the output is what you say, and yet it is ignored.

 

Is there any way to debug it? 

I would start by looking at the master's log.  If you're not already running the master with --debug output turned on, then I would do that, too, to make the log more informative.  Likewise, look at the agent's log.  In particular, make sure the agent successfully retrieved a fresh catalog from the master -- it will complain if it cannot do so, but it may then proceed to use a cached catalog.

John

[shlo....@gmail.com]

Thanks for the details response.

On Tuesday, June 17, 2014 10:28:01 PM UTC+3, jcbollinger wrote:

Probably.  Hooking up an ENC is not very hard.

If the ENC is not running when a catalog request comes in then most likely the ENC it is not properly configured.  Supposing that you are running in agent / master mode, the ENC needs to be set up on the master.  There, the 'node_terminus' configuration parameter must be set to "exec" and the 'external_nodes' configuration parameter must be set to the absolute pathname of the ENC script.

Furthermore, the script must be accessible and executable by the master, which typically does not run as a privileged user.  Not only the script, but also all directories in the path to it must be readable and executable/traversable by the master.  File permissions and other access controls may come into play here.

Also, once it is properly configured, you need to restart the master in order for the new configuration to be noticed.

 

All done already as you describe.

 

It is highly unlikely that the master is running the ENC successfully, the output is what you say, and yet it is ignored.
 

 

Is there any way to debug it? 

I would start by looking at the master's log.  If you're not already running the master with --debug output turned on, then I would do that, too, to make the log more informative.  Likewise, look at the agent's log.  In particular, make sure the agent successfully retrieved a fresh catalog from the master -- it will complain if it cannot do so, but it may then proceed to use a cached catalog.

I will try that.

John

Thanks a lot! 

[shlo....@gmail.com]

Hi,

I still can make the ENC work on my puppet server.

General:
Puppet server: 3.3.1 Puppet agent: 2.7.25
The server run Passenger on Apache, so the puppetmaster run from apache and not by himself.

I have in /etc/puppet/puppet.conf :
[master]
    mode_terminus = exec
    external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=http://user:pass@localhost:3000 /usr/share/puppet-dashboard/bin/external_node

script details:
The file external_node  contain perl, all  the directories and the external_node itself have the permission 755.

I did:
*  When I run the perl script locally  (as a regular script from the command line – not with puppet) on the puppet server it run okay and give the YAML output I sent in my previous emails.
*  looking for errors in messages log and apache log – nothing look suspicious. But the warning:
     -   [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
      -  YAML in network requests is deprecated and will be removed in a future version.
*  Also tried to make syntax error in  my perl script external_node – got no error in the logs.

What make me think maybe:
* puppet it not running external_node script at all.
* ENC have problems when it Passenger & Apache?

Thanks for all the people who answer me.

[RichTea]

On 18 June 2014 14:10, <shlo....@gmail.com> wrote:

Hi,

I still can make the ENC work on my puppet server.

General:
Puppet server: 3.3.1 Puppet agent: 2.7.25
The server run Passenger on Apache, so the puppetmaster run from apache and not by himself.

I have in /etc/puppet/puppet.conf :
[master]
    mode_terminus = exec
   

​​

external_nodes =

​​

/usr/bin/env

​​

PUPPET_DASHBOARD_URL=http://user:pass@localhost:3000

​​

/usr/share/puppet-dashboard/bin/external_node

​Once again why do you have all that in your external_nodes​

 

?

  eg what is ​"/usr/bin/env" doing, what is "​PUPPET_DASHBOARD_URL=http://

user:pass@localhost:3000" doing for you?

Why not just ​external_nodes = ​/usr/share/puppet-dashboard/

bin/external_node

Is your external_nodes script executable by puppet?

Did you add the "node: <certname>" bit to your YAML output as suggested?

--

Ritchie

script details:
The file external_node  contain perl, all  the directories and the external_node itself have the permission 755.

I did:
*  When I run the perl script locally  (as a regular script from the command line – not with puppet) on the puppet server it run okay and give the YAML output I sent in my previous emails.
*  looking for errors in messages log and apache log – nothing look suspicious. But the warning:
     -   [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
      -  YAML in network requests is deprecated and will be removed in a future version.
*  Also tried to make syntax error in  my perl script external_node – got no error in the logs.

What make me think maybe:
* puppet it not running external_node script at all.
* ENC have problems when it Passenger & Apache?

Thanks for all the people who answer me.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/4c5248b8-35c3-4013-8556-1fd7626682f0%40googlegroups.com.

[shlo....@gmail.com]

I tried also just /usr/share/puppet-dashboard/bin/external_node it act the same way.

I add node:, I hope I did it in the right way because I did not find it in the document - also not work.

If I'm not adding any 'node: ', It should work for all the nodes the same. Am I right? or it must appear in the output?

Also I have another warning that I did not mention in my previous email, which I consider as warning:

 Gem::Specification#installation_path called from /usr/share/puppet-dashboard/config/../vendor/rails/railties/lib/rails/vendor_gem_source_index.rb:93.

 NOTE: Gem::Specification#installation_path is deprecated, use base_dir. It will be removed on or after 2011-10-01.

I tried to solve this warning in the past - with no success.

Thank you.

[jcbollinger]

On Wednesday, June 18, 2014 8:10:38 AM UTC-5, shlo....@gmail.com wrote:

Hi,

I still can make the ENC work on my puppet server.

General:
Puppet server: 3.3.1 Puppet agent: 2.7.25

That's a potential problem, but not likely the one you're currently fighting.  The master provides as much compatibility with older agents as it can, but that's so much older an agent (in version number terms) that you're pushing the envelope.  Moreover, even the master is a little old -- current is 3.6.2.

 

The server run Passenger on Apache, so the puppetmaster run from apache and not by himself.

I have in /etc/puppet/puppet.conf :
[master]
    mode_terminus = exec
    external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=http://user:pass@localhost:3000 /usr/share/puppet-dashboard/bin/external_node

Per the docs, the 'external_nodes' property should be the full path to the ENC executable.  Note well: "path to the executable" not "command line". Puppet will run that executable -- probably directly, not via a shell -- passing it the node's certname as the only argument.

 

script details:
The file external_node  contain perl, all  the directories and the external_node itself have the permission 755.

I did:
*  When I run the perl script locally  (as a regular script from the command line – not with puppet) on the puppet server it run okay and give the YAML output I sent in my previous emails.
*  looking for errors in messages log and apache log – nothing look suspicious. But the warning:
     -   [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
      -  YAML in network requests is deprecated and will be removed in a future version.
*  Also tried to make syntax error in  my perl script external_node – got no error in the logs.

What make me think maybe:
* puppet it not running external_node script at all.

That seems a reasonable conclusion.

 

* ENC have problems when it Passenger & Apache?

No.  But perhaps either

1. Puppet is not using the config file you think it's using, or
2. you have not properly restarted the master to get it to read its updated configuration.
   

Under Passenger, the config file Puppet uses is affected by your config.ru.  In particular, therein you need to be sure to set the appropriate --confdir and --vardir.  The default config.ru shipped with Puppet should be correct for a default Puppet installation, but if you installed in a different location or rolled your own config.ru then perhaps the problem is there.  It's worth having a look in any case.


John

[Matthew Burgess]

On 18 Jun 2014 14:10, <shlo....@gmail.com> wrote:
>
>
> Hi,
>
> I still can make the ENC work on my puppet server.
>
> General:
> Puppet server: 3.3.1 Puppet agent: 2.7.25
> The server run Passenger on Apache, so the puppetmaster run from apache and not by himself.
>
> I have in /etc/puppet/puppet.conf :
> [master]
>     mode_terminus = exec

??? That should be node_terminus not mode_terminus. 

Matt

[shlo....@gmail.com]

It's work.

It was spelling problem, I change it now to:

node_terminus = exec.

And it work when 'external_nodes =' with or without the ' /usr/bin/env PUPPET_DASHBOARD_URL=http://user:pass@localhost:3000'
I also delete 'node: <certname>' from my script.

Now when I start httpd, after changing the puppet.conf, I get in the system message file:

Config file /etc/puppet/puppet.conf changed; triggering re-parse of all config files.

But I start getting in the puppet server, system message file:

puppet-agent[23124]: Failed to apply catalog: getaddrinfo: Name or service not known
puppet-agent[23124]: Could not send report: getaddrinfo: Name or service not known

Sorry & I'm very appreciate your help, Thanks to everyone!

Q: If I upgrade puppet, should I backup something. Can it make my puppet server/Passenger stop working?

[jcbollinger]

On Thursday, June 19, 2014 2:48:19 AM UTC-5, shlo....@gmail.com wrote:

It's work.

It was spelling problem, I change it now to:

node_terminus = exec.

Good.

 

And it work when 'external_nodes =' with or without the ' /usr/bin/env PUPPET_DASHBOARD_URL=http://user:pass@localhost:3000'

Surprising.  You should anyway omit that bit.

 

I also delete 'node: <certname>' from my script.

Fine. I'm not sure what that's supposed to be for, either.  I don't think it's required.

 

Now when I start httpd, after changing the puppet.conf, I get in the system message file:

Config file /etc/puppet/puppet.conf changed; triggering re-parse of all config files.

Ok.

 

But I start getting in the puppet server, system message file:

puppet-agent[23124]: Failed to apply catalog: getaddrinfo: Name or service not known
puppet-agent[23124]: Could not send report: getaddrinfo: Name or service not known

It looks like the agent is trying to contact the master via a machine name that it cannot resolve.  Since this is happening on the machine that hosts the master, and you've been mucking with the Puppet configuration on that machine, it seems likely that you changed something there to create this problem (or maybe Puppet itself did, at your behest).

In particular, unless your machine can resolve the hostname "puppet" (to the address of the master), the [agent] section of your puppet.conf should contain a 'server' setting that identifies the puppetmaster machine by a resolvable name that appears on the master's certificate.  Normally, the master's FQDN is a good choice.

 

Sorry & I'm very appreciate your help, Thanks to everyone!

Q: If I upgrade puppet, should I backup something. Can it make my puppet server/Passenger stop working?

It's always wise to make backups before updating critical components.  Any update you perform carries some risk of breakage.

With that said, Puppet bugfix updates (x.y.n to x.y.m) are pretty safe.  You should not expect any breakage around such updates unless you happen to be relying on buggy behavior (which is never safe).  Puppet feature updates (x.n.y to x.m.z) are generally safe.  They are not supposed to cause breakage of any documented behavior, but regressions occasionally happen.

You should already have your manifests (including third-party modules) and data under revision control, in which case you don't need any additional backup of those.  For the most protection, you should also back up your master's data directory, especially all the SSL certificates.  Losing those may require you to generate new certificates for all your clients, which can really ruin your day.


John