Puppet Templates - RFC

[Henrik Lindberg]

In 3.5.0, the ARM-3 Puppet Templates feature is available when using
--parser future.

The ARM-3 text has been updated to reflect the implementation.

Since already published Armature documents are difficult to comment on,
the same text is available for comments here:
https://docs.google.com/a/puppetlabs.com/document/d/1JzDDvSYD3qgeTr6a3H0Ur8GHNVLepyJMLid9YHLgYVg/edit#

If you just want a short introduction, you can read my blog post
about Puppet Templates here:
http://puppet-on-the-edge.blogspot.se/2014/03/templating-with-embedded-puppet.html

I hope you enjoy the new functionality, and I am looking forward to
feedback if you like the new features EPP templates bring.

The idea is to make Puppet Templates a standard feature in Puppet 4.0

Regards
- henrik

[Andy Parker]

There hasn't been any discussion on this as far as I can see. The functionality is in the 3.5.0 version that we had to pull, and it will also be in 3.5.1 that we should be releasing soon. If anyone has any comments please let us know. The design of this is still open for change until we reach puppet 4, where the new language system will become the default and any more changes to this will become more difficult.

- henrik

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lg7hfc%24amu%241%40ger.gmane.org.
For more options, visit https://groups.google.com/d/optout.

--

Andrew Parker

an...@puppetlabs.com

Freenode: zaphod42

Twitter: @aparker42

Software Developer

Join us at PuppetConf 2014, September 22-24 in San Francisco

Register by May 30th to take advantage of the Early Adopter discount —save $349!

[Joshua Hoblitt]

I see the "Non Goals" section at the top of the arm but...

The biggest frustration I have with ERB is when you end up needing
nested flow control logic. As example, to walk through a
hash/array/etc. nested inside some conditional logic. The result is an
eye gouging mix of nested <% ... %> and <% end %> tags that are either
unindented or you have have to salt all the tags with -'s to try and
properly eat the whitespace.

Here's a snippet from a template I was working on yesterday:

```
<% if megaraid_device and megaraid_device != '' and
@megaraid_adapters and @megaraid_adapters.to_i > 0 -%>
<%- unless @megaraid_physical_drives_sata.nil? -%>
<%- @megaraid_physical_drives_sata.split(/,/).sort.each do
|drive| -%>
<%= megaraid_device %> -d sat+megaraid,<%= drive.to_i -%>
<%- if megaraid_options %><%= ' ' + megaraid_options %><% end %>
<%- end -%>
<%- end -%>
<%- unless @megaraid_physical_drives_sas.nil? -%>
<%- @megaraid_physical_drives_sas.split(/,/).sort.each do
|drive| -%>
<%= megaraid_device %> -d megaraid,<%= drive.to_i -%>
<%- if megaraid_options %><%= ' ' + megaraid_options %><% end %>
<%- end -%>
<%- end -%>
<% end -%>
```

It's completely reasonable to argue that some of the boolean logic and
sanitation (the input variables are mostly facts) could be lifted up
into the DSL and simplified to boolean flag variables that are passed to
the template. Under EPP such flag variables could at least be scoped.
However, that would not eliminate the nested conditional logic /tag mess.

It would be nice, under some circumstances, to have a function/method to
"print" from the puppet mode sections of the template similar to this
functionality for rails:

http://api.rubyonrails.org/classes/ActionView/Helpers/TextHelper.html#method-i-concat

-Josh

--

>> email to puppet-dev+...@googlegroups.com.

[Trevor Vaughan]

I usually just make a big blob at the top that builds my output.

<%

output = []

if @stuff

 output << 'foo'

end

output << 'bar'
-%>

<%= output.join("\n") %>

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/534D7CBD.5030905%40cpan.org.

For more options, visit https://groups.google.com/d/optout.

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[John Bollinger]

You are complaining about a problem partially of your own making: you break up the Ruby a lot more than you need to do.  Obviously you can do it like that, but why?

Here's an alternative that I like a lot better:

<%
  if megaraid_device and megaraid_device != '' and
       @megaraid_adapters and @megaraid_adapters.to_i > 0

     option_str = (megaraid_options ? ' ' + megaraid_options : '')
     unless @megaraid_physical_drives_sata.nil?

        @megaraid_physical_drives_sata.split(/,/).sort.each do |drive|
-%>
<%= megaraid_device %> -d sat+megaraid,<%= drive.to_i %><%= option_str %>
<%
        end
    end
    unless @megaraid_physical_drives_sas.nil?
      @megaraid_physical_drives_sas.split(/,/).sort.each do |drive|
-%>
<%= megaraid_device %> -d megaraid,<%= drive.to_i %><%= option_str %>
<%
      end
    end
  end
-%>

Or I don't like this one as much, but it's still better than what you started with:

<% if megaraid_device and megaraid_device != '' and @megaraid_adapters and @megaraid_adapters.to_i > 0 -%>

<%    unless @megaraid_physical_drives_sata.nil? -%>

<%        @megaraid_physical_drives_sata.split(/,/).sort.each do |drive| -%>

<%= megaraid_device %> -d sat+megaraid,<%= drive.to_i -%>

<%            if megaraid_options %><%= ' ' + megaraid_options %><% end %>
<%        end -%>
<%    end -%>
<%    unless @megaraid_physical_drives_sas.nil? -%>
<%        @megaraid_physical_drives_sas.split(/,/).sort.each do |drive| -%>

<%= megaraid_device %> -d megaraid,<%= drive.to_i -%>

<%            if megaraid_options %><%= ' ' + megaraid_options %><% end %>
<%        end -%>
<%    end -%>
<% end -%>

John

[John Bollinger]

On Tuesday, April 15, 2014 1:01:47 PM UTC-5, Andy Parker wrote:

There hasn't been any discussion on this as far as I can see. The functionality is in the 3.5.0 version that we had to pull, and it will also be in 3.5.1 that we should be releasing soon. If anyone has any comments please let us know. The design of this is still open for change until we reach puppet 4, where the new language system will become the default and any more changes to this will become more difficult.

On the main topic of the thread, then: I like the EPP idea on multiple levels.  In particular,

• learning curve.  Puppet users can employ templating without needing to learn any Ruby.
• design. Decoupling of templates from the Puppet execution environment and provisions for defining a (semi-)formal interface to each template are both big wins.
  
• familiarity.  For those of us who already know ERB (or any of several other templating languages, for that matter), the new-style templates are cast in the same mold.

Good work!


John

[Erik Dalén]

Another way to improve it would be if the % and-or <> flags were passed to the ERB builder. % enables Ruby code processing for lines beginning with %. <> omit newline for lines starting with <% and ending in %>. They would allow you to skip a lot of the <%- -%> tags.

Obviously this would be a backwards incompatible change for the regular template function. But for EPP we could do it.

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/3c46642a-5ede-42e5-ac69-7a8ac438ea43%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Erik Dalén

[Joshua Hoblitt]

On 04/15/2014 12:28 PM, John Bollinger wrote:
>
> You are complaining about a problem partially of your own making: you break

> up the Ruby a lot more than you need to do. Obviously you *can* do it like
> that, but why?

I find it's difficult to comprehend nested code intermixed with lines
that start with markup tags.

> Here's an alternative that I like a lot better:

I think this is arguably worse because it's visually very difficult to
line up the the start and end of code blocks.

> <%
> if megaraid_device and megaraid_device != '' and
> @megaraid_adapters and @megaraid_adapters.to_i > 0
> option_str = (megaraid_options ? ' ' + megaraid_options : '')
> unless @megaraid_physical_drives_sata.nil?
> @megaraid_physical_drives_sata.split(/,/).sort.each do |drive|
> -%>
> <%= megaraid_device %> -d sat+megaraid,<%= drive.to_i %><%= option_str %>
> <%
> end
> end
> unless @megaraid_physical_drives_sas.nil?
> @megaraid_physical_drives_sas.split(/,/).sort.each do |drive|
> -%>
> <%= megaraid_device %> -d megaraid,<%= drive.to_i %><%= option_str %>

This is still a nightmare to edit [without breaking whitespace/etc.].

> <%
> end
> end
> end
> -%>

This is an illustrative example of the issue I was describing. It's a
"design smell", in my opinion.

-Josh

--

[Joshua Hoblitt]

+1

This looks like a good work around. I will have to give this a try.

-Josh

--

[John Bollinger]

On Tuesday, April 15, 2014 6:28:58 PM UTC-5, Joshua Hoblitt wrote:

On 04/15/2014 12:28 PM, John Bollinger wrote:
>
> You are complaining about a problem partially of your own making: you break
> up the Ruby a lot more than you need to do.  Obviously you *can* do it like
> that, but why?

I find it's difficult to comprehend nested code intermixed with lines
that start with markup tags.

Fair enough.  Though I can't quite keep myself from suggesting that it's then ERB in general that you dislike.

 

I agree that there's a design smell here.  A template such as this one -- mostly Ruby with only a few bits of interspersed template text -- would be better implemented as a custom function.


John

[Andy Parker]

On Tue, Apr 15, 2014 at 1:20 PM, Erik Dalén <erik.gus...@gmail.com> wrote:

Another way to improve it would be if the % and-or <> flags were passed to the ERB builder. % enables Ruby code processing for lines beginning with %. <> omit newline for lines starting with <% and ending in %>. They would allow you to skip a lot of the <%- -%> tags.

I'm not too keen on this. I think that I'd rather keep the single way to do it instead of branching out into various aliases for functionality.

I am considering Joshua's request for the concat function. It would provide a way to emit text from within the embedded code in the template, and might help in some situations. But it another case where it adds a bit of special stuff to do something that that already be achieved with a little refactoring.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAAAzDLc_ZZxfrSfb5XjK6XTLqttcZDXFKG5Zc2a0BrLKMVs_3A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Aaron Stone]

On Wed, Apr 16, 2014 at 11:53 AM, Andy Parker <an...@puppetlabs.com> wrote:

On Tue, Apr 15, 2014 at 1:20 PM, Erik Dalén <erik.gus...@gmail.com> wrote:

Another way to improve it would be if the % and-or <> flags were passed to the ERB builder. % enables Ruby code processing for lines beginning with %. <> omit newline for lines starting with <% and ending in %>. They would allow you to skip a lot of the <%- -%> tags.

I'm not too keen on this. I think that I'd rather keep the single way to do it instead of branching out into various aliases for functionality.

I am considering Joshua's request for the concat function. It would provide a way to emit text from within the embedded code in the template, and might help in some situations. But it another case where it adds a bit of special stuff to do something that that already be achieved with a little refactoring.

ERB already has a concat function natively with Ruby string syntax. Given this template:

<%

foo = 'FOO'

arg = '99'

options = 'WHAT'

%>

<%= "#{foo} -d bar #{arg.to_i} -- #{options}" %>

Yields:

FOO -d bar 99 -- WHAT

  

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANhgQXvka6QBhdx2unib-6pW7nspTbeT0VEkKZ_ZVG3SpJOD1Q%40mail.gmail.com.

[Joshua Hoblitt]

On 04/16/2014 11:53 AM, Andy Parker wrote:
> I am considering Joshua's request for the concat function. It would provide
> a way to emit text from within the embedded code in the template, and might
> help in some situations. But it another case where it adds a bit of special
> stuff to do something that that already be achieved with a little
> refactoring.

In fairness, a lot of the data structure walking that we see going on in
ERB templates today is because there's no way form of iteration in the
DSL without enabling the future parser. Doing so just isn't an option
for modules going to the forge. The situation will be a lot better when
the lowest common denominator for puppet versions that need to be
supported has a form of data structure iteration.

Still, I think some sort of echo/print/concat like functionality is
justified in templates/views. In particular for the case where you want
to pass a [complex] template into a module as a parameter and not have
to fiddle with the internal data mangling logic.

-Josh

--

[Henrik Lindberg]

On 2014-16-04 21:55, Aaron Stone wrote:
> On Wed, Apr 16, 2014 at 11:53 AM, Andy Parker <an...@puppetlabs.com
> <mailto:an...@puppetlabs.com>> wrote:
> I am considering Joshua's request for the concat function. It would
> provide a way to emit text from within the embedded code in the
> template, and might help in some situations. But it another case
> where it adds a bit of special stuff to do something that that
> already be achieved with a little refactoring.
>
>
> ERB already has a concat function natively with Ruby string syntax.
> Given this template:
>
> <%
> foo = 'FOO'
> arg = '99'
> options = 'WHAT'
> %>
> <%= "#{foo} -d bar #{arg.to_i} -- #{options}" %>
>
> Yields:
> FOO -d bar 99 -- WHAT
>

EPP supports <%= > tags as well which is exactly what a "concatenate
expression result as string" would do.

I can also imagine an epp_concat, that raises an error if called outside
of an epp context, and otherwise does the same as <%= >, but it is
really not required (it is just less noisy).

- henrik

[Felix Frank]

On 04/16/2014 08:53 PM, Andy Parker wrote:
> Another way to improve it would be if the % and-or <> flags were
> passed to the ERB builder. % enables Ruby code processing for lines
> beginning with %. <> omit newline for lines starting with <% and
> ending in %>. They would allow you to skip a lot of the <%- -%> tags.
>
>
> I'm not too keen on this. I think that I'd rather keep the single way to
> do it instead of branching out into various aliases for functionality.

Strongly agreed.

Templates are challenging to some users as is. Making it so that
different templates require different parameters to the template()
function call (or differently flavored functions) in the manifest to
work properly will make it (much) harder.

[Simon Marechal]

I think this is a great addition, as it will simplify a lot the thinking about templates : no arbitrary side effects (like you have with arbitrary ruby code), and, more importantly, explicit parameters. This last feature is just great.

I am however wondering why this is so, as it seems like such a bad idea ?

"There is however no protection against users creating resources inside
the template (nor if they do this via function calls to `create_resources`).
There is also no protection against realizing/collecting resources."

I also disagree with :

If someone chooses to use these questionable expressions inside a template, there is no real harm;

But that's probably just a question of taste :)

[Henrik Lindberg]

On 2014-22-04 10:59, Simon Marechal wrote:
> I think this is a great addition, as it will simplify a lot the thinking
> about templates : no arbitrary side effects (like you have with
> arbitrary ruby code), and, more importantly, explicit parameters. This
> last feature is just great.

Great. Glad you like that.

> I am however wondering why this is so, as it seems like such a bad idea ?
>
> "There is however no protection against users creating resources inside
> the template (nor if they do this via function calls to
> `create_resources`).
> There is also no protection against realizing/collecting resources."
>
>

We can add such protection - the cost is performance. While we do like
protection against mistakes, there is a limit to what can be done to
stop those that insist on using oddball constructs as there is always
some other way to do strange things.

> I also disagree with :
>
> If someone chooses to use these questionable expressions inside a
> template, there is no real harm;
>
>
> But that's probably just a question of taste :)
>

Yeah, we really do not want people to do stupid things - hence pointing
out that it is stupid (although it will not kill you). (People do things
like this today, producing template text and calling create resources
from ruby because they need both the resources and the text...).

Regards
- henrik

[Trevor Vaughan]

I'd like to say that I certainly don't want to do stupid things but I'll all about doing strange things that work for what I need to get done.

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lj5ofk%24kqb%241%40ger.gmane.org.

For more options, visit https://groups.google.com/d/optout.

--

[Henrik Lindberg]

On 2014-22-04 15:26, Trevor Vaughan wrote:
> I'd like to say that I certainly don't want to do stupid things but I'll
> all about doing strange things that work for what I need to get done.
>

I realize, that what I wrote was a bit unclear. I did mean that 'calling
create resource from within the template code itself (which is the
equivalence of using resource expressions in a puppet based template)'
is just as horrible irrespective of language. That was the bad thing to
do that I pointed out.

Totally understandable that working around problems requires using some
strange solutions from time to time.

- henrik

> Trevor
>
>
> On Tue, Apr 22, 2014 at 8:48 AM, Henrik Lindberg

> <henrik....@cloudsmith.com <mailto:henrik....@cloudsmith.com>>
> wrote:

> send an email to puppet-dev+unsubscribe@__googlegroups.com
> <mailto:puppet-dev%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/__msgid/puppet-dev/lj5ofk%24kqb%__241%40ger.gmane.org
> <https://groups.google.com/d/msgid/puppet-dev/lj5ofk%24kqb%241%40ger.gmane.org>.
>
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.

>
>
>
>
> --
> Trevor Vaughan
> Vice President, Onyx Point, Inc
> (410) 541-6699

> tvau...@onyxpoint.com <mailto:tvau...@onyxpoint.com>

>
> -- This account not approved for unencrypted proprietary information --
>

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoVAe6N_qY%2BVLGzaN223RKGZh5rj0VxRN4cTavKMrRObMA%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoVAe6N_qY%2BVLGzaN223RKGZh5rj0VxRN4cTavKMrRObMA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Erik Dalén]

Is there any syntax validation tool available for this? Will something like "puppet parser validate --parser=future" just work on epp files?

I'm looking for some equivalent to "erb -x file.erb | ruby -c".

- henrik

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lg7hfc%24amu%241%40ger.gmane.org.

For more options, visit https://groups.google.com/d/optout.

--
Erik Dalén

[Andy Parker]

On Mon, May 12, 2014 at 7:13 AM, Erik Dalén <erik.gus...@gmail.com> wrote:

Is there any syntax validation tool available for this? Will something like "puppet parser validate --parser=future" just work on epp files?

I'm looking for some equivalent to "erb -x file.erb | ruby -c".

There isn't one as far as I know. The parser needs to be switched to a different mode and so the validate command can't handle EPP. I've opened PUP-2531 to track the request.

 

On 17 March 2014 20:12, Henrik Lindberg <henrik....@cloudsmith.com> wrote:

In 3.5.0, the ARM-3 Puppet Templates feature is available when using --parser future.

The ARM-3 text has been updated to reflect the implementation.

Since already published Armature documents are difficult to comment on, the same text is available for comments here: https://docs.google.com/a/puppetlabs.com/document/d/1JzDDvSYD3qgeTr6a3H0Ur8GHNVLepyJMLid9YHLgYVg/edit#

If you just want a short introduction, you can read my blog post
about Puppet Templates here: http://puppet-on-the-edge.blogspot.se/2014/03/templating-with-embedded-puppet.html

I hope you enjoy the new functionality, and I am looking forward to feedback if you like the new features EPP templates bring.

The idea is to make Puppet Templates a standard feature in Puppet 4.0

Regards
- henrik

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lg7hfc%24amu%241%40ger.gmane.org.
For more options, visit https://groups.google.com/d/optout.

--
Erik Dalén

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAAAzDLc7cj8ECEWZu%2BeJxshtxpNEn07Sfb92G8O%2Bt%2BEHCtK60Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--