Re: [Puppet-dev] Defining a standard mechanism for template specialization

[Trevor Vaughan]

In this particular scenario, I tend to lean toward having a 'content' variable in my class or define.

class foo (  $content => template('foo/awesome.erb') ) { ... }

Someone wants to override that:

class { 'foo':

  content => template('bar/my_awesome.erb')
}

I'm not sure if that covers 100% of the cases that you're concerned with, but it tends to work well for me as a pattern.

Looking at many of the Forge modules, the main issue appears to me to be that the authors are solving problem X and don't want to create a module that handles X^Y configuration options due to time, cost, and manpower issues (all of which are completely understandable). Also, the effort required to understand and extend an elegant module may be more than people are willing to put in to solve what is, otherwise, a 30 minute problem.

Finally, you run into the situation where someone did take the time to create a completely awesome module but it's so granular that using quite a few of them like that trends toward compiled graph bloat and code complexity.

Hope this helps.

Trevor

On Fri, May 17, 2013 at 11:23 AM, Jeff Goldschrafe <je...@holyhandgrenade.org> wrote:

There are a few tickets floating around, like #7635, that deal with the inability to specialize templates in standard Puppet. Among other things, this is a limitation that makes it somewhat difficult for Forge module authors to distribute truly reusable modules without providing a cadre of non-interoperable custom functions to address this concern (see: multi_source_template). These tickets propose very specific solutions in terms of semantics. I'd like to open a broader feature discussion about how templates can/should be specialized in Puppet manifests, if people think they should be specialized at all. (I'm primarily concerned with templates, but there's probably no reason this discussion couldn't also apply to sourced files as well.)

I'm focused on the following three stories that I've seen come up on lists/IRC from time to time:

• As a user, I want to provide different versions of a template depending on some characteristic of the system so I don't need to encode this differentiation logic into every manifest individually.
• As a module author, I want users to be able to specialize their own templates without having to modify my module.
• As a Puppet administrator in an environment where templates/data are change-controlled differently than manifest code, I want to provide users the ability to produce new templates so they can make necessary changes without altering any manifest logic.
  

--

As an example/starting point, here is a brief summary of my ideal implementation of this in Puppet:

Most web servers provide a content-negotiation implementation where different filenames are evaluated based on some characteristic of the request/requestor. Typically, this is done using the Accept-Language header, where the browser has specified that they would like a representation of the content specific to their language, if that representation is available. If the browser requests index.html and specifies Accept-Language: fr, the web server can reply with index.html.fr if that specific version is available, or index.html if it is not. Many people are already using this model manually with file sources, providing ["puppet:///modules/mymodule/myfile.${osfamily}", "puppet:///modules/mymodule/myfile"]. I'd love to see a less verbose alternative that a) does this automatically based on configuration data rather than code, and that b) works for templates.

At the same time, Puppet already provides hierarchical lookup features for other forms of data via Hiera. Though Hiera can be used (somewhat verbosely) for roughly this purpose, it seems natural to provide a similar capability for template specialization as well.

Implementation discussions may tie into ARM-8 (Data in Modules) hierarchy specification and the discussion on that ARM should be kept in mind.

Thoughts?

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To post to this group, send email to puppe...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[Daniel Pittman]

On Fri, May 17, 2013 at 8:23 AM, Jeff Goldschrafe

<je...@holyhandgrenade.org> wrote:
>
> There are a few tickets floating around, like #7635, that deal with the
> inability to specialize templates in standard Puppet.

[...]

> Many people are already using this model
> manually with file sources, providing
> ["puppet:///modules/mymodule/myfile.${osfamily}",
> "puppet:///modules/mymodule/myfile"]. I'd love to see a less verbose
> alternative that a) does this automatically based on configuration data
> rather than code, and that b) works for templates.

I think the best solution would be to change the `template` function
to select the first entry, rather than concatenate, and allow users to
manually concatenate multiple `template` calls if they wanted it.

I understand the attraction of having an automatic, rather than
manual, model for searching. I don't think it is a realistic goal,
however, because of the wide variety of "search paths" that could be
desired. Explicitly spelling out what is searched seems like the best
balance of flexibility and extensibility to me.

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons

[Ashley Penney]

On Fri, May 17, 2013 at 4:08 PM, Daniel Pittman <dan...@puppetlabs.com> wrote:

I think the best solution would be to change the `template` function
to select the first entry, rather than concatenate, and allow users to
manually concatenate multiple `template` calls if they wanted it.

I understand the attraction of having an automatic, rather than
manual, model for searching.  I don't think it is a realistic goal,
however, because of the wide variety of "search paths" that could be
desired.  Explicitly spelling out what is searched seems like the best
balance of flexibility and extensibility to me.

For what it's worth, this is something I would like to see.  I found the concatenation to be more of a pain than anything else, especially since the concat module came about.  Selecting the first entry would be an overall improvement.

[Daniel Pittman]

I was thinking of either of these two when I said concatenation:

$one = template(...)
$two = template(...)
notice("${one}${two}") # concat!

$three = concat($one, $two) # with the obvious implementation

...but, yeah, the module also satisfies. :)

Basically, the language already allows you to do string pasting
without too much hardship. Might as well do that the same way you do
other string pasting.

[Jeff Goldschrafe]

A $content variable works great in certain instances, but there are others where this particular abstraction falls apart and I've been unable to determine a better one without a sane mechanism for template specialization. For example, if you're creating a defined type that contains a file resource, and you want the template to use variables of that defined type instance (such as $name/$title, for example) this isn't an easy thing to accomplish if you're calling template() from within a different scope in order to pass in content. In the case of this particular problem, the only semi-sane way to approach it is to create a wrapper defined type, which has a really bad code smell to me. And even then, you'd better hope you only need the variables actually passed into the type, and not any derived variables ($whatever_real, etc.).

I'm not sure I buy the point about the wide variety of paths to be searched making this a very difficult problem -- this is exactly what we're doing today with Hiera, and it *is* making it explicit what is searched. There are two components to template specialization that need to be handled separately:

1. Global lookup (defined by site administrators)
2. Local lookup (defined by module authors, like Hiera data in ARM-8)

Site administrators and module authors can then use whatever lookup orders they want ad have them coexist happily.

[Henrik Lindberg]

On 2013-17-05 8:23, Jeff Goldschrafe wrote:
> There are a few tickets floating around, like #7635, that deal with the
> inability to specialize templates in standard Puppet. Among other
> things, this is a limitation that makes it somewhat difficult for Forge
> module authors to distribute truly reusable modules without providing a
> cadre of non-interoperable custom functions to address this concern
> (see: multi_source_template). These tickets propose very specific
> solutions in terms of semantics. I'd like to open a broader feature
> discussion about how templates can/should be specialized in Puppet
> manifests, if people think they should be specialized at all. (I'm
> primarily concerned with templates, but there's probably no reason this
> discussion couldn't also apply to sourced files as well.)
>
> I'm focused on the following three stories that I've seen come up on
> lists/IRC from time to time:
>

> * As a user, I want to provide different versions of a template

> depending on some characteristic of the system so I don't need to
> encode this differentiation logic into every manifest individually.

> * As a module author, I want users to be able to specialize their own

> templates without having to modify my module.

> * As a Puppet administrator in an environment where templates/data are

There are features in ARM-3 (Puppet Templates) that will help with
several of the issues. One such feature is the ability to pass
parameters to the template, and that a template can nest other templates.

https://github.com/puppetlabs/armatures/blob/master/arm-3.puppet_templates/puppet_templates.md#passing-arguments-to-the-template

The ARM-8 (Data in Modules) is also important and work is underway to
define a successor to Hiera ("Hiera 2") with a more general binding scheme.

Puppet Templates is implemented and can be tried out (information in the
ARM).

regards
- henrik