Newbie question regarding the file resource

[Fabien Delpierre]

Hi folks,
I'm super new to Puppet but decently experienced with Chef, I'm trying to apply my Chef logic to some things I'm trying to do in Puppet but I've hit what is probably a simple snag. And perhaps I shouldn't be trying to apply Chef logic, but hear me out first!

Firstly, I'm following along John Arundel's book, Puppet 3 Beginner's Guide. It's had me create a module to do my own things, as well as a separate module that just installs Nginx, and now it's trying to set up a basic website by providing a virtual host file and writing it to /etc/nginx/sites-enabled. The way it's having me do that is to invoke the file resource directly from within my nginx class/module. I know that will work, but that seems like heresy.

In my Chef logic, I would create a cookbook just to set up my website, have it invoke an external cookbook to install Nginx, and then simply drop the Nginx config file in the right spot, like so:

include_recipe "nginx::default"

cookbook_file "/var/www/mysite/index.html"
cookbook_file "/etc/nginx/sites-enabled/mysite.conf"

That separates the process of just installing Nginx (which my simpleton Nginx class is doing just fine) from setting up the website.

What I have so far in Puppet looks like this:

mysite
-files
--mysite.conf

-manifests
--default.pp
--nodes.pp

-modules
--nginx
---manifests
----init.pp

The book wants me to add a files directory under nginx and put the mysite.conf there. But in my logic, the nginx class is an external dependency -- or at least, going forward, I would actually use the "official" Nginx module and call it from my custom module somehow, so assuming the Nginx module is its own entity that I cannot change, I'd have to find a way to write mysite.conf directly from my custom module.

My nodes.pp looks like this:

node 'sensei-debian7' {

  include nginx

  file { "/var/www/mysite/index.html":
    content => "Hello?\n",
  }

  file { ["/var/www", "/var/www/mysite"]:
    ensure => directory
  }

  file { "/etc/nginx/sites-enabled/default":
    source => 'puppet:///sensei/mysite.conf',
    notify => Service['nginx'],
  }
}

That source attribute above is my problem. sensei is the name of my learning module. I'm using Vagrant for testing, and when I run vagrant provision, Puppet complains that it cannot find the path:
==> debian7: Error: /Stage[main]/Main/Node[sensei-debian7]/File[/etc/nginx/sites-enabled/default]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///sensei/mysite.conf

Like I said, the book would like me to put the mysite.conf in modules/nginx/files/mysite.conf and have the following source line:
source => 'puppet:///modules/nginx/mysite.conf',
I'm sure that that would work, but since it feels wrong to do it that way, I'm trying to write that line correctly so that it fetches the mysite.conf directly from my module, without going to the Nginx module at all.
I've tried multiple things:
puppet:///modules/sensei/mysite.conf
puppet:///sensei/mysite.conf
puppet:///files/mysite.conf (I figured that wouldn't work since I understand Puppet adds the files/ in those instances)
puppet:///mysite.conf
and others. But nothing works.

I hope this makes sense.
So, I don't know if I'm going about this all wrong or if I just need to find the right syntax for that source line. If this is all wrong, then what's the right way of doing it?
Let's assume that, later on, I might move my Nginx module to its own Git repo and invoke it from my Sensei module using a Puppetfile/Librarian.

Thanks for reading, and any pointers would be appreciated!

[Peter Berghold]

I think I understand what you are driving at, and your logic seems good to me.  When I was doing web hosting for people I had a class that handled installing and configuring Apache and a separate module called "sites" that handled setting up the web sites using the apache::vhost define that contained the logic to set up a VirtualHost definition file and create the DocRoot directory. 

The difference was I was using templates and synthesizing the configuration file based on some parameters passed to the define.

HTH.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/6e3e084d-700d-4d84-ba89-0e27524aec3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Felix Frank]

On 07/20/2015 06:17 PM, Fabien Delpierre wrote:
> I've tried multiple things:
> puppet:///modules/sensei/mysite.conf

Well that's nice, but earlier in your mail, you stated that mysite.conf
is in the 'myfiles' module. The URL you need is

puppet:///modules/myfiles/mysite.conf

Your reasoning is sound for the most part, but don't create a 'myfiles'
module. That's the actual no-no. If you create a module that installs
files for the 'mysite' vhost, call it mysite or something along those lines.

HTH,
Felix

[Fabien Delpierre]

Peter, Felix,
Thank you for your answers!

I guess my improperly worded question was about figuring out whether Puppet has a concept similar to what Chef calls a wrapper cookbook.

Felix, the "sensei" is my actual folder name, I just substituted for generic names and may have forgotten to subst one or two occurrences.
To be clear, I have two modules:
- The "main" one I'm using for my testing, called "sensei".
- Within it, I have a subfolder called "modules", containing an "nginx" module.
By now I actually moved the nginx module to its own individual Git repo, and added a Puppetfile with the dependency. That works but hasn't helped with my problem, though.

If I place my file in sensei/files/mysite.conf and try to invoke it using this source:
puppet:///modules/sensei/mysite.conf
It doesn't work.

[Fabien Delpierre]

Figured it out! The issue was not with my Puppet code, but simply with my Vagrantfile.

I had the following:

config.vm.provision :puppet do |puppet|
  puppet.manifests_path = "manifests"
  puppet.module_path = "modules"
  puppet.working_directory = "/vagrant"
  puppet.facter = {
  }
end

I edited the third line like so:

config.vm.provision :puppet do |puppet|
  puppet.manifests_path = "manifests"
  puppet.module_path = ["..", "modules"]
  puppet.working_directory = "/vagrant"
  puppet.facter = {
  }
end

And now, using source => 'puppet:///modules/sensei/mysite.conf' is working fine. Makes sense!

[jcbollinger]

On Monday, July 20, 2015 at 9:45:12 PM UTC-5, Fabien Delpierre wrote:

Peter, Felix,
Thank you for your answers!

I guess my improperly worded question was about figuring out whether Puppet has a concept similar to what Chef calls a wrapper cookbook.

I know nothing more about "wrapper cookbooks" than I can deduce from their name and your code, but they sound like they may have some similarity with the Puppet pattern called "Roles & Profiles", conceived by Craig Dunn.  You can find a lot of information about it around the web, but here's the original description: http://www.craigdunn.org/2012/05/239/.

As for your original problem, I'm inclined to agree with you that it's wrong to put your site-specific nginx config file in a general-purpose nginx module.  The need to modify a module itself to customize it for your site violates its modularity, and it will bite you as soon as soon as your site requires two unrelated nginx servers.

On the other hand, the nginx module should manage the standard nginx configuration files, because it needs to ensure that the correct relationships are maintained between that file and other nginx components (its Package, the nginx Service, etc.), and because other code shouldn't need to know details of how nginx config files are structured.

The trick, then, is that you want the nginx module to manage the file to have whatever content (and other properties) you need it to have, without making that content itself belong to the module.  This is separation of data from code, which is a well established best practice for designing Puppet modules and manifest sets.  The typical approach to this problem involves giving one or more of the classes of the nginx module class parameters by which you can provide the necessary data (the appropriate 'content' or 'source' for the managed file, for instance).  You then provide the data via automated data binding through Hiera.

Under some circumstances, you may be able to split configuration between files managed by the general-purpose module, and files managed by site-specific classes.  This generally happens when the file to be managed supports an 'include'-like directive to incorporate the contents of other files into itself by reference, or when the component being managed automatically reads all files from a given directory.  Under these circumstances it is often to your advantage to wrap the native resources involved in a defined type provided by the GP module.  That way all the needed relationships and internal plumbing can be handled by the defined type, and the user just needs to provide the site-specific details.


John

[Fabien Delpierre]

On Tuesday, July 21, 2015 at 9:46:56 AM UTC-4, jcbollinger wrote:

I know nothing more about "wrapper cookbooks" than I can deduce from their name and your code, but they sound like they may have some similarity with the Puppet pattern called "Roles & Profiles", conceived by Craig Dunn.  You can find a lot of information about it around the web, but here's the original description: http://www.craigdunn.org/2012/05/239/.

I skimmed through this, was interested to find out more about roles and will have to read it more in-depth, but that doesn't seem similar to a wrapper cookbook. But basically, now that I've got it working, I've attained what I would call a "wrapper module" that works fine, using logic similar to what I would do with Chef.
 

As for your original problem, I'm inclined to agree with you that it's wrong to put your site-specific nginx config file in a general-purpose nginx module.  The need to modify a module itself to customize it for your site violates its modularity, and it will bite you as soon as soon as your site requires two unrelated nginx servers.

Absolutely. I imagine that the book I'm working with was urging me to do that for the sake of simplicity, probably because its target audience is people who are not already familiar with a configuration management tool, so the way they had me do it set off my internal alarms since I'm not in that audience and just need to get familiar with the tool rather than bigger-picture, overarching concepts of CM.
 

On the other hand, the nginx module should manage the standard nginx configuration files, because it needs to ensure that the correct relationships are maintained between that file and other nginx components (its Package, the nginx Service, etc.), and because other code shouldn't need to know details of how nginx config files are structured.

Oh definitely, that's also what I'd do. My nginx module does not currently manage the nginx config file, but it would if I were actually writing a proper nginx module. I'm just practicing, at this point. Seems like the next big thing I need to get cozy with is Hiera.

Thanks!