Ability to deploy a "skeleton" directory

[Jean-Baptiste Barth]

Hi,

I'd like to be able to deploy a skeleton directory through puppet, i.e. puppet deploys it only if it does not exist.

I tried this :

file {"/path/to/this/directory":

  source => "puppet:///files/path/to/the/skeleton",

  ensure => directory,

  recurse => true,

  owner => auser,

  group => agroup

}

But I have 2 problems for the moment :

- if I don't set owner/group, it takes uid/gid on the puppetmaster, which does not exist on the node ; if I do set these options, the directory is scanned each time and all files are chown'ed to this user/group, which is absolutely not desired behaviour in my case (puppet should not change ownership for files it doesn't manage in my case...)

- I'd like puppet to let user change some files in the skeleton as they want, i.e. skip this directory if it just exists

Do you have an idea about how I could do that ?

Thanks,

-- 

Jean-Baptiste

[Silviu Paragina]

On 25.06.2010 12:00, Jean-Baptiste Barth wrote:
> Hi,
>
> I'd like to be able to deploy a skeleton directory through puppet,
> i.e. puppet deploys it only if it does not exist.
>
> I tried this :
> file {"/path/to/this/directory":
> source => "puppet:///files/path/to/the/skeleton",
> ensure => directory,
> recurse => true,
> owner => auser,
> group => agroup
> }
>
> But I have 2 problems for the moment :
> - if I don't set owner/group, it takes uid/gid on the puppetmaster,
> which does not exist on the node ; if I do set these options, the
> directory is scanned each time and all files are chown'ed to this
> user/group, which is absolutely not desired behaviour in my case
> (puppet should not change ownership for files it doesn't manage in my
> case...)

Change the ownership of the files on the server (but this implies that
the users exists on the server too). Second what puppet version? There
were some bugs in older versions that made puppet hash all the files in
a folder if the folder was maintained by puppet, maybe that's the
problem in your case. Also try changing the hash property of file. If
you want that folder under a certain user you should allow puppet to
ensure that.

> - I'd like puppet to let user change some files in the skeleton as
> they want, i.e. skip this directory if it just exists

This is not possible, and would break the way puppet works if it were to
be implemented. Usually local changes can be integrated through the use
of the concat module (see puppet forge). But I don't know your real
usage of the files so it may be different. If you post what you need to
do maybe someone would give you a better idea of achieving just that.

You may try to create a file/fact for puppet to indicate that it
shouldn't touch that folder on that computer. Other than that no ideas,
but again post a more concrete usage scenario. Also maybe puppet isn't
the right tool for you since it works by ensuring some specific
settings, not allowing local changes. Maybe you need to generalize you
manifests instead of allowing local changes.

Welcome to puppet ;) ,
Silviu

[Patrick Mohr]

On Jun 25, 2010, at 2:00 AM, Jean-Baptiste Barth wrote:

- I'd like puppet to let user change some files in the skeleton as they want, i.e. skip this directory if it just exists

Do you have an idea about how I could do that ?

Do you mean you want the users to be able to change all the files?  I just pushed the directory using an exec with a "creates".  I think I used a combination of wget and tar.

[Jean-Baptiste Barth]

2010/6/25 Silviu Paragina <sil...@paragina.ro>

Change the ownership of the files on the server (but this implies that the users exists on the server too). Second what puppet version? There were some bugs in older versions that made puppet hash all the files in a folder if the folder was maintained by puppet, maybe that's the problem in your case. Also try changing the hash property of file. If you want that folder under a certain user you should allow puppet to ensure that.

 

I think the version is 0.25.4, but I'm not really sure. Anyway, it's the package provided on our Debian, I cannot change it easily. Thanks for the hash idea. 

This is not possible, and would break the way puppet works if it were to be implemented. Usually local changes can be integrated through the use of the concat module (see puppet forge). But I don't know your real usage of the files so it may be different. If you post what you need to do maybe someone would give you a better idea of achieving just that.

Sorry I was not very clear. I didn't mean "change the files puppet manages", but more "change some files under the skeleton directory". I have 2 use cases where it could be  interesting :

1- deploy skeleton directories directly ; users could put their own vimrc, bashrc, etc. on the puppetmaster and they would find it everywhere on the platform. As my platform has many OS inside, it would be less complicated than deploying automount homedirs or anything like that.

2- deploy empty applications ; for instance rails apps, but not only ; administrator defines the app main parameters on the puppetmaster, it is deployed on the platform, then some manual configurations can be done on files that puppet does not manage (let's say database is not configured by puppet, puppet just puts a database.yml.example in the target server and the administrator defines the database.yml by hand before running db:migrate and starting the app) 

You may try to create a file/fact for puppet to indicate that it shouldn't touch that folder on that computer. Other than that no ideas, but again post a more concrete usage scenario. Also maybe puppet isn't the right tool for you since it works by ensuring some specific settings, not allowing local changes. Maybe you need to generalize you manifests instead of allowing local changes.

Welcome to puppet ;) ,
Silviu

Hope it is clearer now. Thank you for taking the time to answer my puppet-newbie problems ;)

-- 

Jean-Baptiste 

[Jean-Baptiste Barth]

2010/6/26 Patrick Mohr <kc7...@gmail.com>

Do you mean you want the users to be able to change all the files?  I just pushed the directory using an exec with a "creates".  I think I used a combination of wget and tar.

Sounds like a good idea, maybe it will be much more simple in my case. Anyway, see my other answer, I did not mean users should be able to change files puppet manages, I was not very clear in my first post, sorry.

Thanks,

-- 

Jean-Baptiste

[Patrick Mohr]

On Jun 26, 2010, at 12:32 AM, Thomas Bellman wrote:

> Jean-Baptiste Barth wrote:
>
>> I'd like to be able to deploy a skeleton directory through puppet, i.e. puppet deploys it only if it does not exist.

> [...]

>> But I have 2 problems for the moment :
>> - if I don't set owner/group, it takes uid/gid on the puppetmaster, which does not exist on the node ; if I do set these options, the directory is scanned each time and all files are chown'ed to this user/group, which is absolutely not desired behaviour in my case (puppet should not change ownership for files it doesn't manage in my case...)
>> - I'd like puppet to let user change some files in the skeleton as they want, i.e. skip this directory if it just exists
>

> The second problem you can solve by using the 'replace => false'
> parameter to the file type. However, if the user *removes* one
> of the files in the skeleton directory, then Puppet will download
> it again, so maybe it isn't quite enough.
>
> And even with 'replace => false', Puppet will manage ownerships
> and modes of all files.
>
> One solution is to develop a custom fact that gets set to true
> if the target directory already exists, and not set if it doesn't.
> Then you can do
>
> if $dir_opt_foo_data_exists != "true" {
> file {
> "/opt/foo/data": source => ..., recurse => true, ...;
> }
> }
>
> This doesn't scale very well to many directories, though, as you
> need one custom fact for each such directory. Another way is to
> do it with an exec:
>
> exec {
> opt-foo-data:
> command => "wget -r http://.../ && chown -R auser:agroup data",
> cwd => "/opt/foo", path => "/bin:/usr/bin",
> creates => "/opt/foo";
> }
>
> Note the use of the parameter 'creates => "/opt/foo"', which will
> make Puppet only run the command if /opt/foo doesn't already exist.
>
> The disadvantage is that you need to configure and run a web server
> (or ftp server) somewhere as well.

I usually do it this way (using wget and tar), but you could push the directory somewhere else using puppet and put "cp -a" in the exec.

[John Lyman]

> - I'd like puppet to let user change some files in the skeleton as they
> want, i.e. skip this directory if it just exists

You should use "recurse => remote" to solve this issue. See
http://projects.reductivelabs.com/issues/1469 and
http://projects.reductivelabs.com/issues/1700 for more info. You
might also want to set "checksum => none" (new in version 0.25.5) to
cut down on some of the reporting noise (any time a user would change
a file that puppet was not managing, the directory checksum would
change and I would get an email report of it.)

> - if I don't set owner/group, it takes uid/gid on the puppetmaster, which
> does not exist on the node ; if I do set these options, the directory is
> scanned each time and all files are chown'ed to this user/group, which is
> absolutely not desired behaviour in my case (puppet should not change
> ownership for files it doesn't manage in my case...)

In the specific case of home directories, I created a definition that
adds the user and then sets the ownership of the home directory to
that user's uid and gid. Remote recurse will also solve this problem
in general.