Concatenating files

[Jean-Baptiste Quenot]

Hi there,

I want one of my configuration files to be a concatenation of various
files, and tried those various ways:

1) ERB template: this looks not viable as one cannot include a file or
another template in an ERB template. That's a pity.

2) generate() function:

content => generate("/bin/sh", "-c", "cat file1 file2")

Or expressed differently:

content => generate("/bin/cat", "file1", "file2")

There is no example for generate() in the function reference, so I had
to search the mailing-lists to find out this construct. I can't edit
the FunctionReference wiki, so I thought I'd let you know.

And of course this only works if you make sure to use "content"
instead of "source". I had hard times figuring out that Puppet was
trying to resolve another source when I used "source" instead of
"content". Hence an example in the wiki would be nice.

3) Writing your own function

I put my custom function in lib/puppet/parser/functions/cat.rb:

module Puppet::Parser::Functions
newfunction(:cat, :type => :rvalue) do |args|
lines1 = IO.read(args[0])
lines2 = IO.read(args[1])
lines1 + lines2
end
end

This works, but I'd like to improve my function to recognize the
puppet:// protocol. How can one achieve that?

Also, I tried to call self.interp.newfile(<FILENAME>) like suggested
in the documentation at
http://reductivelabs.com/trac/puppet/wiki/WritingYourOwnFunctions#AccessingFiles
But this was blocking the configuration retrieval, so I finally
removed it from my custom function. Is the documentation outdated in
this regard?

Your help and comments will be appreciated.

Thanks in advance,
--
Jean-Baptiste Quenot
http://caraldi.com/jbq/blog/

[Mike Renfro]

On 3/4/2008 5:16 AM, Jean-Baptiste Quenot wrote:

> I want one of my configuration files to be a concatenation of various
> files, and tried those various ways:

I got a version of a concatenated_file definition from David Schmitt's
repository long ago, not sure if it's still there. Here's the one I'm
using now:

| define concatenated_file ($dir, $mode = 0644, $owner = root, $group =
root)
| {
| file {
| $name:
| ensure => present,
| checksum => md5,
| mode => $mode,
| owner => $owner,
| group => $group;
| }
|
| exec { "find ${dir} -maxdepth 1 -type f ! -name '*puppettmp'
-print0 | sort -z | xargs -0 cat > ${name}":
| refreshonly => true,
| subscribe => File[$dir],
| alias => "concat_${name}",
| }
| }

Usage:

| file {
| "/etc/puppet/files/apps/pam_listfile/users.conf.ch204b.d":
| ensure => directory,
| checksum => mtime;
| }
| concatenated_file {
| "/etc/puppet/files/apps/pam_listfile/users.conf.ch204b":
| dir => "/etc/puppet/files/apps/pam_listfile/users.conf.ch204b.d";
| }

Place a bunch of alphanumerically-ordered files in a directory, and
they'll all get concatenated into one big config file. I also made a
concatenated_file_uniq definition that culls out duplicate lines (not
useful for most configuration files, but very useful for maintaining
lists of authorized users, where an individual user might gain
authorization from enrolling in a class, filling out a form, or whatever).

--
Mike Renfro / R&D Engineer, Center for Manufacturing Research,
931 372-3601 / Tennessee Technological University

[David Schmitt]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 04 March 2008, Mike Renfro wrote:
> On 3/4/2008 5:16 AM, Jean-Baptiste Quenot wrote:
> > I want one of my configuration files to be a concatenation of various
> > files, and tried those various ways:
>
> I got a version of a concatenated_file definition from David Schmitt's
> repository long ago, not sure if it's still there.

Sure it is. The current version is at:

http://git.black.co.at/?p=module-common;a=blob;f=manifests/defines/concatenated_file.pp;h=e3dbe23f22477cadd4980343d892d72e87953c02;hb=HEAD

It now has grown header and footer support. Also I have the _part define which
takes the same params as the main define.

> I also made a
> concatenated_file_uniq definition that culls out duplicate lines (not
> useful for most configuration files, but very useful for maintaining
> lists of authorized users, where an individual user might gain
> authorization from enrolling in a class, filling out a form, or whatever).

Nice idea! Two questions though: why didn't you just add this as a
parameter "uniq => true" and why did i get a patch? :)

Regards, DavidS

- --
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHzbMa/Pp1N6Uzh0URAp8gAJ9ch8HXxRub2Ub4jdbJmc50hUbURQCfc3n0
66Gwjx6own3M/nyCdetXIj4=
=DZVd
-----END PGP SIGNATURE-----