generate function

3,415 views
Skip to first unread message

Craig White

unread,
Aug 10, 2011, 12:40:11 PM8/10/11
to Puppet Users
Seems I don't quite understand how it's supposed to work

At the moment, I have it inside a 'file' resource

content => generate("/etc/puppet/scripts/ldap-add-host.sh $fqdn admins_all"),

Thus I want to execute the command
/etc/puppet/scripts/ldap-add-host.sh

and pass parameters
$1 = $fqdn
$2 = admins_all (though there will be other groups to pass in the future)

the error it gives me is...

Could not retrieve catalog from remote server: Error 400 on SERVER: Generators can only contain alphanumerics, file separators, and dashes at /etc/puppet/modules/ldap/manifests/configure.pp:93 on node ubuntu.ttinet

documentation on the generate function is sort of sparse and I am not able to deduce the structure of an acceptable command but I gather that spaces are not allowed.

How do I structure this?

--
Craig White ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ craig...@ttiltd.com
1.800.869.6908 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ www.ttiassessments.com

Need help communicating between generations at work to achieve your desired success? Let us help!

Daniel Pittman

unread,
Aug 10, 2011, 12:52:09 PM8/10/11
to puppet...@googlegroups.com
On Wed, Aug 10, 2011 at 16:40, Craig White <craig...@ttiltd.com> wrote:

> Seems I don't quite understand how it's supposed to work
> At the moment, I have it inside a 'file' resource
>
> content => generate("/etc/puppet/scripts/ldap-add-host.sh $fqdn admins_all"),

generate("/etc/puppet/scripts/ldap-add-hosh.sh", $fqdn, "admins_all")

Daniel
--
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons

Craig White

unread,
Aug 10, 2011, 1:56:30 PM8/10/11
to puppet...@googlegroups.com

On Aug 10, 2011, at 9:52 AM, Daniel Pittman wrote:

> On Wed, Aug 10, 2011 at 16:40, Craig White <craig...@ttiltd.com> wrote:
>
>> Seems I don't quite understand how it's supposed to work
>> At the moment, I have it inside a 'file' resource
>>
>> content => generate("/etc/puppet/scripts/ldap-add-host.sh $fqdn admins_all"),
>
> generate("/etc/puppet/scripts/ldap-add-hosh.sh", $fqdn, "admins_all")

----
that worked great but brings me to a place I can't figure out.

A file resource doesn't have 'unless' or 'onlyif' and thus it seems to execute every time.

An exec resource doesn't have 'content' but does have command and 'unless' so it would seem exec would be a better way to go but...

# Puppet maintained file /etc/puppet/deployment_files/ldap_admins_all
exec{"/etc/puppet/deployment_files/ldap_admins_all":
command => ['/bin/touch /etc/puppet/deployment_files/admins_all', generate("/etc/puppet/scripts/ldap-add-host.sh", $fqdn, "admins_all")],
unless => "/bin/ls -l /etc/puppet/deployment_files/ldap_admins_all",
require => Class["mod_puppet::deployment_files"],
}

gives me the error...

err: Failed to apply catalog: private method `split' called for #<Array:0x4873be8>

and I could pass an array to 'file' resource using content but not 'exec' resource using command

;-(

Craig

Daniel Pittman

unread,
Aug 10, 2011, 2:14:25 PM8/10/11
to puppet...@googlegroups.com

Yeah. So, that is a totally awful error message. The `command` needs
to be a string, not an array, but the exec type isn't checking that,
it just fails trying to call the Ruby `split` method on it. Which
doesn't work. :)

That said, it isn't entirely clear to me what you are trying to do.

The `generate` function runs a command on the Puppet master while the
catalog is being compiled, but an `exec` resource is run on the
client. Generally, you use `generate` to allow you to query an
external data source.

Is your `ldap-add-host.sh` script doing that query, or does it
actually create things?

If the later, is it idempotent - will it do the same thing if you run it twice?

Finally, you need multiple `exec` resources (or a shell script) if you
want to run multiple commands. Generally, best to be explicit about
that, so you would separate the touch and the running of your script.


Anyway, at a guess, `generate` is not at all what you want to do, and
you should try and forget about it. Instead, make the
`ldap-add-host.sh` script create the lock on disk, and then just run
that internally.

Craig White

unread,
Aug 10, 2011, 2:31:50 PM8/10/11
to puppet...@googlegroups.com

----
what I am trying to do is execute a shell script on the puppetmaster... essentially add 'host' attribute to specific ldap users. That's why the command has parameters...

shellscript HOSTNAME GROUP

the script is more than capable of getting the users from GROUP, adding host attribute HOSTNAME to each of the users but it must run on the puppetmaster, not on puppet clients which is why I am using the generate function. So in answer to your question, my ldap-add-host.sh script is actually creating things.

yes, it is idempotent - I can run it and run it and it will always do the same thing but and if uid=craig already has 'host' ubuntu.ttinet, it will simply move on if I try to add it again. I could almost live with that except that if I manually remove 'host' ubuntu.ttinet from uid=craig, the next pass it will add it again so I need some method of tracking it so therefore I was trying to use 'unless' which is only available in an exec resource, not a file resource. I suppose if I had no alternative, I could maintain a list on the puppetmaster of which hosts have already been added to which groups and abort if it has already been done.

Craig


Daniel Pittman

unread,
Aug 10, 2011, 2:50:50 PM8/10/11
to puppet...@googlegroups.com

OK.

> the script is more than capable of getting the users from GROUP, adding host attribute HOSTNAME to each of the users but it must run on the puppetmaster, not on puppet clients which is why I am using the generate function. So in answer to your question, my ldap-add-host.sh script is actually creating things.
>
> yes, it is idempotent - I can run it and run it and it will always do the same thing but and if uid=craig already has 'host' ubuntu.ttinet, it will simply move on if I try to add it again. I could almost live with that except that if I manually remove 'host' ubuntu.ttinet from uid=craig, the next pass it will add it again so I need some method of tracking it so therefore I was trying to use 'unless' which is only available in an exec resource, not a file resource. I suppose if I had no alternative, I could maintain a list on the puppetmaster of which hosts have already been added to which groups and abort if it has already been done.

OK. So, yeah. `generate` doesn't do what you want: functions don't
take parameters of any sort, let alone resource level metaparameters.

You will need to implement all your logic in the script you invoke
from generate, so that it will avoid doing things twice when called
with the same arguments.


...and if you are wondering why this seems so hard? This really isn't
something that Puppet is designed to support. Generally, modifying
external data sources from Puppet like you are trying to do isn't
really the way we approach things. Better, we feel, to modify the
external data source and then draw read-only from that into the
manifest.

So, rather than calling generate to modify LDAP, instead modify LDAP
and have code in your manifest to do whatever stuff when the LDAP
changes have been applied.

You can do it the way you are trying, more or less, but you really
don't get much help from the tool.s

Craig White

unread,
Aug 10, 2011, 5:13:09 PM8/10/11
to puppet...@googlegroups.com

----
I modified my shell script to keep track of what has already been added to LDAP and return a constant result if already added. It's really neat how standard output from the generate command ends up as the content of a 'file' on the client but yes, the 'generate' command will run each time the puppet client runs because the only way it can decide what the content actually is going to be is by running again and comparing with the file that is (or isn't) on the client already. In my methodology, I already had a directory /etc/puppet/deployment_files for keeping track of things so it was a simple task to have file resources for things that really aren't a file on a normal server but triggers for actions (or inaction) by puppet.

I never really wondered why or even thought it was hard to accomplish this because I definitely understand that processing is really targeted at the client level.

But I am sure you understand the desire to integrate more than just the clients - in this case, we are dealing with a large set of knowns...
- the new host/node
- ldap configuration from the first puppet run that implements host based access control
- the members of specific groups
- the distribution of sudoers 'include' files for these groups

and thus the only missing link was the ability to actually be able to log into these hosts was for admins to have the host attributes set for the new hosts. This was separately scripted and I just needed the missing link and that was provided to me by generate... not perfect, not exactly how I would have wanted it but definitely workable.

Thanks for the help - it was invaluable and made this a relatively simple task

Craig

Scott Smith

unread,
Aug 10, 2011, 11:04:07 PM8/10/11
to puppet...@googlegroups.com
Sounds like a job better suited for ENC/inventory. IMO.


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.


Reply all
Reply to author
Forward
0 new messages