Decision: Near future of resource expressions
171 views
Skip to first unread message
Andy Parker
Jul 24, 2014, 8:32:13 PM7/24/14
to puppe...@googlegroups.com
Howdy,
Henrik, David, Erik, John, and others have been having some pretty epic conversations around resource expressions, precedence, order of evaluation, and several other topics. What kicked all of that off was us looking for some feedback on decisions we were making for the Puppet 4 language about how resource overrides, defaults, and so on actually work (or don't in some cases). I think we've finally reached some decisions!
Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#
Lots of information in there, so here is the summary.
The principles behind the decisions:
1. Only make changes that have a high likelihood of *not* needing to be backed out later.
2. Strict or explicit is better than lax or implicit. It uncovers issues and keeps you from lying to yourself.
3. Puppet 3 has already stacked up a lot of changes. Do not break manifests unless we really have to.
4. Let's not break manifests and then have to break them in almost the same way once we start working on a new catalog system.
There are three kinds of resource expression that we have to deal with:
1. Resource instantiation
2. Resource defaults
3. Resource overrides
Looking forward, I think it is highly likely that the catalog system that we'll be working on during puppet 4 will be some sort of production (rules) system. In that kind of a world, resource instantiation likely remains as is, but defaults and overrides will end up having to change quite a bit, if not in syntax, at least in semantics.
DECISION ONE
Resource defaults and Resource overrides will be left untouched.
Decision one follows from principles 3 and 4. In the discussions it became clear that changing when defaults or overrides are applied, the scope of defaults, or anything else about them was going to cause a lot of problems. Puppet's master branch changed resource defaults to follow the same scoping rules as variables. That change will be reverted.
DECISION TWO
Resource instantiations are value producing expressions
The expression based grammar that puppet 4 will be based on changed almost everything into a general expression, which allowed a lot of composition that wasn't possible before. This didn't change resource expressions. Resource expressions could not be assigned ($a = notify {hi:}). That is being changed. This removes several odd corners in the grammar and makes it all more consistent. It is also highly unlikely that it would be removed later (principle 1). The value of a resource expression is a reference to the created resource, or an array of references if there is more than one.
QUESTION: should the value always be an array of references? That would make it much more predictable.
DECISION THREE
Resource instantiation expressions will not be allowed in dangerous locations
Once resource expressions can be placed anywhere there are a few places where they would actually just do more harm than good (principle 2). One example is as a parameter default (define a($b = notify {hi:}) {}).
DECISION FOUR
The LHS of a resource *instantiation* expression can be an expression
What?!? This means you can do:
$a = notify
$a { hi: }
Once again, in clearing up odd cases in the grammar this is opened up to us. This is a very powerful feature to have available. Since this is very useful and fits well into the grammar I don't see this being a temporary thing that would then have to go away later (principle 1).
DECISION FIVE (how many of these are there?)
A resource with a title of default provides the default parameter values for other resources in the same instantiation expression.
Thanks to David Schmitt for this idea!
Since we aren't going to change the behavior of resource default expressions (Notify { ... }) it seems like there needs to be something done to provide a better, safer way of specifying defaults. This will allow:
notify {
default: message => hi;
bye: }
The result will be a resource of type Notify with title bye and message hi. It is highly unlikely that this will go away (principle 1) as it is syntactic sugar for specifying the parameters for every resource.
DECISION SIX
There will be a splat operator for resource instantiation expressions
To make the default resources (decision five) really useful there needs to be a way to reuse the same values across multiple defaults. The current, dangerous, semantics of resource default expressions skirt this issue by making defaults part of the (dynamic) evaluation scope. In order to make the default resources nearly as useful but much safer, we need to add a way to allow reuse of defaults across multiple resource instantiation expressions explicitly (principle 2).
$owner_mode = { owner => andy, mode => '777' } # gotta make it secure
file { default: *=> $owner_mode;
'/home/andy/.bashrc': ;
'/home/andy/.ssh/id_rsa': ;
}
file { '/etc/passwd': *=> $owner_mode }
As a side note, do you see what can now be done?
$a = notify
$b = hi
$c = { message => bye }
$a { $b: *=> $c }
DECISION SEVEN
undef is not allowed as a title
Not much to say here. notify { undef: } fails (or anything that evaluates to undef)
DECISION EIGHT
An array as a title expands to individual resource instantiation expressions with titles of the elements of the array.
This isn't really too far off from the current semantics, no real change here. It is only to call out that we are formalizing that as the semantics. An empty array ends up being a noop (no resources instantiated). An array that contains undef will produce an error (see decision seven). The value default can be an element of the array and will produce the default section for the resources being instantiated (as pointless as that seems since they will all have the same body).
DECISION NINE
Decisions two through eight do not apply to resource default or resource override expressions.
Just to make it clear that decision one still holds.
CONCLUSION
I think that covers it all. This will be reflected by a revert to some code, modifying the grammar, adding some new evaluation capabilities, including tests, and updating the specification. All of this is falling under PUP-501, PUP-511, and PUP-2898 in some way shape or form.
This email was to record the decisions; make them public; double check that Henrik, Joshua and I all had the same understanding of them; and give another chance to everyone to weigh in.
I did have one question that I uncovered as I was writing this up. Some feedback on that would be great as well.
Andrew Parker
Freenode: zaphod42
Twitter: @aparker42
Software DeveloperRegister by May 30th to take advantage of the Early Adopter discount —save $349!
Henrik Lindberg
Jul 24, 2014, 9:05:01 PM7/24/14
to puppe...@googlegroups.com
A title expression must result in a String value, or Array of String
values. No regular expressions, hashes, booleans, numbers etc.
No magic turning a title into a string if it is not.
> Freenode: zaphod42
> Twitter: @aparker42
> Software Developer
>
> *Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September
> Twitter: @aparker42
> Software Developer
>
> 22-24 in San Francisco*
> /Register by May 30th to take advantage of the Early Adopter discount
> <http://links.puppetlabs.com/puppetconf-early-adopter> //—//save $349!/
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
--
Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/
Spencer Krum
Jul 24, 2014, 9:32:15 PM7/24/14
to puppe...@googlegroups.com
I'm not sure if this is the correct time to mention this, but I wonder if you considered arrays of hashes in decision eight?
I guess they are not really arrays of hashes but whatever this is:
[
'/root/file1' => {'owner' => 'root'},
'/root/file1' => {'owner' => 'nibz'},
]
Right now we often use arrays of hashes with the create_resources function when we need to specify parameters. This is similar to the effect of how arrays passed into resources as title behave.
I think it would be awesome if we could pass what we currently pass into create_resources into resource instantiations.
Thanks,
Spencer
I guess they are not really arrays of hashes but whatever this is:
[
'/root/file1' => {'owner' => 'root'},
'/root/file1' => {'owner' => 'nibz'},
]
Right now we often use arrays of hashes with the create_resources function when we need to specify parameters. This is similar to the effect of how arrays passed into resources as title behave.
I think it would be awesome if we could pass what we currently pass into create_resources into resource instantiations.
Thanks,
Spencer
an email to puppet-dev+unsubscribe@googlegroups.com<mailto:puppet-dev+unsub...@googlegroups.com>.
<https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com
--
Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lqsaff%2438h%241%40ger.gmane.org.
Henrik Lindberg
Jul 25, 2014, 8:40:00 AM7/25/14
to puppe...@googlegroups.com
On 2014-25-07 3:32, Spencer Krum wrote:
> I'm not sure if this is the correct time to mention this, but I wonder
> if you considered arrays of hashes in decision eight?
>
> I guess they are not really arrays of hashes but whatever this is:
>
> [
>
> '/root/file1' => {'owner' => 'root'},
> '/root/file1' => {'owner' => 'nibz'},
> ]
>
> Right now we often use arrays of hashes with the create_resources
> function when we need to specify parameters. This is similar to the
> effect of how arrays passed into resources as title behave.
>
> I think it would be awesome if we could pass what we currently pass into
> create_resources into resource instantiations.
>
You mean like this?
> I'm not sure if this is the correct time to mention this, but I wonder
> if you considered arrays of hashes in decision eight?
>
> I guess they are not really arrays of hashes but whatever this is:
>
> [
>
> '/root/file1' => {'owner' => 'root'},
> '/root/file1' => {'owner' => 'nibz'},
> ]
>
> Right now we often use arrays of hashes with the create_resources
> function when we need to specify parameters. This is similar to the
> effect of how arrays passed into resources as title behave.
>
> I think it would be awesome if we could pass what we currently pass into
> create_resources into resource instantiations.
>
$x = [
'/root/file1' => {'owner' => 'root'},
'/root/file1' => {'owner' => 'nibz'},
]
file { $x: }
'/root/file1' => {'owner' => 'nibz'},
]
When we discuss this, we preferred that the iteration is made explicit.
You can use the same data structure, and do this:
$x.each |$title, $attributes| { file { $title: * => $attributes } }
Since that is much more descriptive. (In fact, that is pretty much what
the implementation of create resources is).
- henrik
> Thanks,
> Spencer
>
>
> On Thu, Jul 24, 2014 at 6:04 PM, Henrik Lindberg
>
> Freenode: zaphod42
> Twitter: @aparker42
> Software Developer
>
> *Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September
> 22-24 in San Francisco*
> /Register by May 30th to take advantage of the Early Adopter
> discount
> <http://links.puppetlabs.com/__puppetconf-early-adopter
> Freenode: zaphod42
> Twitter: @aparker42
> Software Developer
>
> *Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September
> 22-24 in San Francisco*
> /Register by May 30th to take advantage of the Early Adopter
> discount
> <http://links.puppetlabs.com/puppetconf-early-adopter>>
> //—//save $349!/
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from
> it, send
> an email to puppet-dev+unsubscribe@__googlegroups.com
> //—//save $349!/
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from
> it, send
> <mailto:puppet-dev%2Bunsu...@googlegroups.com>
> <mailto:puppet-dev+_...@googlegroups.com
> <mailto:puppet-dev%2Bunsu...@googlegroups.com>>.
>
> To view this discussion on the web visit
> https://groups.google.com/d/__msgid/puppet-dev/__CANhgQXu3HVrWJrTnMgYvbY6%__3DR8B%__3DvVgts2Uqmwjtj6eJRJsH7g%__40mail.gmail.com
> To view this discussion on the web visit
> <https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com>
> <https://groups.google.com/d/__msgid/puppet-dev/__CANhgQXu3HVrWJrTnMgYvbY6%__3DR8B%__3DvVgts2Uqmwjtj6eJRJsH7g%__40mail.gmail.com?utm_medium=__email&utm_source=footer
> <https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
>
>
> --
>
> Visit my Blog "Puppet on the Edge"
> http://puppet-on-the-edge.__blogspot.se/
>
>
> --
>
> Visit my Blog "Puppet on the Edge"
> <http://puppet-on-the-edge.blogspot.se/>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to puppet-dev+unsubscribe@__googlegroups.com
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> <mailto:puppet-dev%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/__msgid/puppet-dev/lqsaff%2438h%__241%40ger.gmane.org
> <https://groups.google.com/d/msgid/puppet-dev/lqsaff%2438h%241%40ger.gmane.org>.
>
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
>
>
>
> --
> Spencer Krum
> (619)-980-7820
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> <mailto:puppet-dev+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CADt6FWNcOa2vaxx_vNn%2BNTX1tmngeF2KEi0mspQcCxUBrh3mYg%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CADt6FWNcOa2vaxx_vNn%2BNTX1tmngeF2KEi0mspQcCxUBrh3mYg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Spencer Krum
Jul 25, 2014, 10:06:06 AM7/25/14
to puppe...@googlegroups.com
Okay. Thanks.
On Fri, Jul 25, 2014 at 5:39 AM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
On 2014-25-07 3:32, Spencer Krum wrote:You mean like this?
I'm not sure if this is the correct time to mention this, but I wonder
if you considered arrays of hashes in decision eight?
I guess they are not really arrays of hashes but whatever this is:
[
'/root/file1' => {'owner' => 'root'},
'/root/file1' => {'owner' => 'nibz'},
]
Right now we often use arrays of hashes with the create_resources
function when we need to specify parameters. This is similar to the
effect of how arrays passed into resources as title behave.
I think it would be awesome if we could pass what we currently pass into
create_resources into resource instantiations.
$x = [file { $x: }
'/root/file1' => {'owner' => 'root'},
'/root/file1' => {'owner' => 'nibz'},
]
When we discuss this, we preferred that the iteration is made explicit. You can use the same data structure, and do this:
$x.each |$title, $attributes| { file { $title: * => $attributes } }
Since that is much more descriptive. (In fact, that is pretty much what the implementation of create resources is).
- henrik
Thanks,
Spencer
On Thu, Jul 24, 2014 at 6:04 PM, Henrik Lindberg
To view this discussion on the web visit
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lqtj6g%24png%241%40ger.gmane.org.
Andy Parker
Jul 25, 2014, 12:25:37 PM7/25/14
to puppe...@googlegroups.com
On Thu, Jul 24, 2014 at 6:04 PM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
DECISION SEVEN AND 3/4On 2014-25-07 2:32, Andy Parker wrote:
DECISION SEVEN
undef is not allowed as a title
Not much to say here. notify { undef: } fails (or anything that
evaluates to undef)
A title expression must result in a String value, or Array of String values. No regular expressions, hashes, booleans, numbers etc.
No magic turning a title into a string if it is not.
I agree in principle, but am worried about the impact. Right now it is somewhat common to do things like:
# bad example but should illustrate the idea
define hosts() {
host { $title[hostname]: ip => $title[ip] }
}
hosts { [{ hostname => andy, ip => '192.168.12.3' },
{ hostname => henrik, ip => '192.168.12.4' }]: }
This ends up creating instances of Hosts with hashes as titles. Often this works out fine...just not always.
an...@puppetlabs.com <mailto:an...@puppetlabs.com>
--
Andrew Parker
*Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September
Freenode: zaphod42
Twitter: @aparker42
Software Developer
22-24 in San Francisco*
/Register by May 30th to take advantage of the Early Adopter discount
<http://links.puppetlabs.com/puppetconf-early-adopter> //—//save $349!/
--
You received this message because you are subscribed to the Google
Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to puppet-dev+unsubscribe@googlegroups.com
<mailto:puppet-dev+unsub...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com
<https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lqsaff%2438h%241%40ger.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Andrew Parker
Register by May 30th to take advantage of the Early Adopter discount —save $349!
John Bollinger
Jul 25, 2014, 4:43:07 PM7/25/14
to puppe...@googlegroups.com
On Thursday, July 24, 2014 7:32:13 PM UTC-5, Andy Parker wrote:
Howdy,Henrik, David, Erik, John, and others have been having some pretty epic conversations around resource expressions, precedence, order of evaluation, and several other topics. What kicked all of that off was us looking for some feedback on decisions we were making for the Puppet 4 language about how resource overrides, defaults, and so on actually work (or don't in some cases). I think we've finally reached some decisions!Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#Lots of information in there, so here is the summary.The principles behind the decisions:1. Only make changes that have a high likelihood of *not* needing to be backed out later.2. Strict or explicit is better than lax or implicit. It uncovers issues and keeps you from lying to yourself.3. Puppet 3 has already stacked up a lot of changes. Do not break manifests unless we really have to.4. Let's not break manifests and then have to break them in almost the same way once we start working on a new catalog system.
Great principles.
There are three kinds of resource expression that we have to deal with:1. Resource instantiation2. Resource defaults3. Resource overridesLooking forward, I think it is highly likely that the catalog system that we'll be working on during puppet 4 will be some sort of production (rules) system. In that kind of a world, resource instantiation likely remains as is, but defaults and overrides will end up having to change quite a bit, if not in syntax, at least in semantics.DECISION ONEResource defaults and Resource overrides will be left untouched.Decision one follows from principles 3 and 4. In the discussions it became clear that changing when defaults or overrides are applied, the scope of defaults, or anything else about them was going to cause a lot of problems. Puppet's master branch changed resource defaults to follow the same scoping rules as variables. That change will be reverted.
I take it you mean that the current syntax for overrides and defaults will continue to be supported and will continue to have the traditional semantics. Do you also mean that the new form of default declarations will also have the same semantics (scope, laziness, ...) as the traditional ones? (I suspect not.)
DECISION TWOResource instantiations are value producing expressions
Love it.
What about collection expressions? Or does their late evaluation present too much of a problem?
QUESTION: should the value always be an array of references? That would make it much more predictable.
How about making the value an array if and only if the title given in the resource instantiation expression is an array? In most cases either you know the type of the title or you don't care; in the odd case that you don't know the type of the title but you do care about the result being an array then you can write
$array = flatten( [ notify {$maybe_array: message => 'hi' } ] )
(requires stdlib).
If you want an array but you have only a single title then you can write either
$array = notify { [ 'adieu' ]: message => 'bye' }
or
$array = [ notify { 'adieu': message => 'bye' } ]
DECISION THREEResource instantiation expressions will not be allowed in dangerous locationsOnce resource expressions can be placed anywhere there are a few places where they would actually just do more harm than good (principle 2). One example is as a parameter default (define a($b = notify {hi:}) {}).
Sounds wise, but even after reading the details in your googledoc I'm not sure I follow the details.
DECISION FOURThe LHS of a resource *instantiation* expression can be an expressionWhat?!? This means you can do:$a = notify$a { hi: }Once again, in clearing up odd cases in the grammar this is opened up to us. This is a very powerful feature to have available. Since this is very useful and fits well into the grammar I don't see this being a temporary thing that would then have to go away later (principle 1).
This one makes me a bit uncomfortable, but I could see it maybe being useful. Are there limitations on the value produced by an expression used as the LHS of a resource instantiation expression? For instance, that it must evaluate to either a string or a (resource) type?
DECISION FIVE (how many of these are there?)A resource with a title of default provides the default parameter values for other resources in the same instantiation expression.Thanks to David Schmitt for this idea!
Yes, this is cool.
DECISION SIXThere will be a splat operator for resource instantiation expressionsTo make the default resources (decision five) really useful there needs to be a way to reuse the same values across multiple defaults. The current, dangerous, semantics of resource default expressions skirt this issue by making defaults part of the (dynamic) evaluation scope. In order to make the default resources nearly as useful but much safer, we need to add a way to allow reuse of defaults across multiple resource instantiation expressions explicitly (principle 2).$owner_mode = { owner => andy, mode => '777' } # gotta make it securefile { default: *=> $owner_mode;'/home/andy/.bashrc': ;'/home/andy/.ssh/id_rsa': ;}file { '/etc/passwd': *=> $owner_mode }
"*=>" looks weird, even for Puppet. What about contracting it to "*>", parallel to the plussignment operator (+>) and to the ordinary value binding operator (=>)?
DECISION EIGHTAn array as a title expands to individual resource instantiation expressions with titles of the elements of the array.This isn't really too far off from the current semantics, no real change here.
Is there any change at all there with respect to what current Puppet already supports? (I don't see any, but you've spooked me.)
John
John Bollinger
Jul 25, 2014, 4:59:43 PM7/25/14
to puppe...@googlegroups.com
On Friday, July 25, 2014 11:25:37 AM UTC-5, Andy Parker wrote:
On Thu, Jul 24, 2014 at 6:04 PM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
DECISION SEVEN AND 3/4On 2014-25-07 2:32, Andy Parker wrote:
DECISION SEVEN
undef is not allowed as a title
Not much to say here. notify { undef: } fails (or anything that
evaluates to undef)
A title expression must result in a String value, or Array of String values. No regular expressions, hashes, booleans, numbers etc.
No magic turning a title into a string if it is not.
I agree in principle, but am worried about the impact. Right now it is somewhat common to do things like:# bad example but should illustrate the ideadefine hosts() {host { $title[hostname]: ip => $title[ip] }}hosts { [{ hostname => andy, ip => '192.168.12.3' },{ hostname => henrik, ip => '192.168.12.4' }]: }This ends up creating instances of Hosts with hashes as titles. Often this works out fine...just not always.
No, that's not common at all, because it doesn't work (or it never used to do, anyway). Resource titles are always strings. Puppet stringifies values of other types (except one level of array) if they are used as resource titles.
This actually comes up from time to time on puppet-users. People try to instantiate a defined type with a hash as its title, and Puppet throws an error when the defined type body tries to apply the element selection operator to the title.
Henrik is suggesting (I think) that the error instead be in the instantiation expression, when it turns out that the title is not a String. I think that's a great idea, because it causes the error to be flagged where it occurs. If you really want to create resources with stringified hashes (or whatever) as titles, then there should be a to_string() function that you can apply to the hashes to show that you really mean it. And with the future parser, you can easily apply it to the array elements in your example if that's what you want to do. It also fits in naturally with the more specific limitation that titles not be undef.
John
Henrik Lindberg
Jul 25, 2014, 6:15:30 PM7/25/14
to puppe...@googlegroups.com
dynamic scoping for resource defaults because the way we changed it on
master makes it incompatible, and it is not replaced by something better.
> Do you also mean that the new form of default declarations
> will also have the same semantics (scope, laziness, ...) as the
> traditional ones? (I suspect not.)
>
expression. it is processed when the resource is instantiated as if
the attribute operations had been written out in long form - only
now they are in a common section, and can come from a hash.
(This only applies to resource instantiation, this is not available in
the 3x style resource override and defaults expressions).
>
> DECISION TWO
>
> Resource instantiations are value producing expressions
>
>
>
> Love it.
>
> What about collection expressions? Or does their late evaluation
> present too much of a problem?
>
CollectionExpression is useless in the language - it has no information
in it of value at that point, only after the catalog is completed.
Although it may be possible to assign the value, it is of a type that
you cannot use for any practical purpose - a pice of dark matter.
>
>
> QUESTION: should the value always be an array of references? That
> would make it much more predictable.
>
>
>
> How about making the value an array if and only if the title given in
> the resource instantiation expression is an array? In most cases either
> you know the type of the title or you don't care; in the odd case that
> you don't know the type of the title but you do care about the result
> being an array then you can write
>
> $array = flatten( [ notify {$maybe_array: message => 'hi' } ] )
>
> (requires stdlib).
>
$array = * notify {$maybe_array: message => 'hi' }
since the splat operator turns something into an array if it appears in
a place where individual elements are meaningless. (i.e. no standardlib
required to array-ify).
> If you want an array but you have only a single title then you can write
> either
>
> $array = notify { [ 'adieu' ]: message => 'bye' }
>
> or
>
> $array = [ notify { 'adieu': message => 'bye' } ]
>
>
> DECISION THREE
>
> Resource instantiation expressions will not be allowed in
> dangerous locations
>
> Once resource expressions can be placed anywhere there are a few
> places where they would actually just do more harm than good
> (principle 2). One example is as a parameter default (define a($b =
> notify {hi:}) {}).
>
>
>
> Sounds wise, but even after reading the details in your googledoc I'm
> not sure I follow the details.
>
simple and I did not clean that up. Basically there are a couple of
places where assignment and resource creation is just weird, and we will
just statically validate expressions in those positions to not contain
any embedded assignments or resource expressions (instantiations,
overrides or defaults).
The places I can think of right now are:
- parameter default value expressions
- case and selector options
Better to validate these than to cause possible mysterious errors caused
by user mistakes. It is not taking away anything of value.
>
> DECISION FOUR
>
> The LHS of a resource *instantiation* expression can be an expression
>
> What?!? This means you can do:
>
> $a = notify
> $a { hi: }
>
> Once again, in clearing up odd cases in the grammar this is opened
> up to us. This is a very powerful feature to have available. Since
> this is very useful and fits well into the grammar I don't see this
> being a temporary thing that would then have to go away later
> (principle 1).
>
>
>
> This one makes me a bit uncomfortable, but I could see it maybe being
> useful. Are there limitations on the value produced by an expression
> used as the LHS of a resource instantiation expression? For instance,
> that it must evaluate to either a string or a (resource) type?
>
or a String that complies with the class name rule.
We reasoned that we already have create_resources in frequent use to
solve real issues, so it is needed. I don't think create_resources is
used simply because you can. Instead, it is because you do not
statically know the type, or you want to decide which attributes to set
based on calculations you made. Both of those require create_resources
function. Now it moves into the language.
> DECISION FIVE (how many of these are there?)
>
> A resource with a title of default provides the default parameter
> values for other resources in the same instantiation expression.
>
> Thanks to David Schmitt for this idea!
>
>
>
> Yes, this is cool.
>
> DECISION SIX
>
> There will be a splat operator for resource instantiation expressions
>
> To make the default resources (decision five) really useful there
> needs to be a way to reuse the same values across multiple defaults.
> The current, dangerous, semantics of resource default expressions
> skirt this issue by making defaults part of the (dynamic) evaluation
> scope. In order to make the default resources nearly as useful but
> much safer, we need to add a way to allow reuse of defaults across
> multiple resource instantiation expressions explicitly (principle 2).
>
> $owner_mode = { owner => andy, mode => '777' } # gotta make it secure
> file { default: *=> $owner_mode;
> '/home/andy/.bashrc': ;
> '/home/andy/.ssh/id_rsa': ;
> }
>
> file { '/etc/passwd': *=> $owner_mode }
>
>
>
> "*=>" looks weird, even for Puppet. What about contracting it to "*>",
> parallel to the plussignment operator (+>) and to the ordinary value
> binding operator (=>)?
>
name => value
Now, you do not have names, and * is often used as a wildcard/splat/
"split it up", implied, the name is picked from the hash, and you may
want to line them up:
file { default:
* => value ;
foo:
mode => '0777' ;
}
Does that make it feel better?
The first proposal was:
title: (expr)
Then, just use the splat operator:
title: *expr
But then it became weird, because it is a special kind of splat that
applies to the entire expression - that felt more confusing.
>
> DECISION EIGHT
>
> An array as a title expands to individual resource instantiation
> expressions with titles of the elements of the array.
>
> This isn't really too far off from the current semantics, no real
> change here.
>
>
>
> already supports? (I don't see any, but you've spooked me.)
>
Current implementation silently skips undef entries, balks at an undef
>
title (not in an array), but allows other types than Strings as
resulting individual titles (with the weird stringification taking place
that we talked about above). Hence, "not too far off". Feel better now?
- henrik
Luke Kanies
Jul 28, 2014, 2:34:41 AM7/28/14
to puppe...@googlegroups.com
On Jul 24, 2014, at 5:32 PM, Andy Parker <an...@puppetlabs.com> wrote:
Howdy,Henrik, David, Erik, John, and others have been having some pretty epic conversations around resource expressions, precedence, order of evaluation, and several other topics. What kicked all of that off was us looking for some feedback on decisions we were making for the Puppet 4 language about how resource overrides, defaults, and so on actually work (or don't in some cases). I think we've finally reached some decisions!Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#
Sorry for the late comments here, but I have a couple of questions and comments.
Lots of information in there, so here is the summary.The principles behind the decisions:1. Only make changes that have a high likelihood of *not* needing to be backed out later.2. Strict or explicit is better than lax or implicit. It uncovers issues and keeps you from lying to yourself.3. Puppet 3 has already stacked up a lot of changes. Do not break manifests unless we really have to.4. Let's not break manifests and then have to break them in almost the same way once we start working on a new catalog system.
Great principles. It would be good to see these paired with what fits into the “we really have to” section of #3.
There are three kinds of resource expression that we have to deal with:1. Resource instantiation2. Resource defaults3. Resource overridesLooking forward, I think it is highly likely that the catalog system that we'll be working on during puppet 4 will be some sort of production (rules) system. In that kind of a world, resource instantiation likely remains as is, but defaults and overrides will end up having to change quite a bit, if not in syntax, at least in semantics.DECISION ONEResource defaults and Resource overrides will be left untouched.Decision one follows from principles 3 and 4. In the discussions it became clear that changing when defaults or overrides are applied, the scope of defaults, or anything else about them was going to cause a lot of problems. Puppet's master branch changed resource defaults to follow the same scoping rules as variables. That change will be reverted.DECISION TWOResource instantiations are value producing expressionsThe expression based grammar that puppet 4 will be based on changed almost everything into a general expression, which allowed a lot of composition that wasn't possible before. This didn't change resource expressions. Resource expressions could not be assigned ($a = notify {hi:}). That is being changed. This removes several odd corners in the grammar and makes it all more consistent. It is also highly unlikely that it would be removed later (principle 1). The value of a resource expression is a reference to the created resource, or an array of references if there is more than one.QUESTION: should the value always be an array of references? That would make it much more predictable.DECISION THREEResource instantiation expressions will not be allowed in dangerous locationsOnce resource expressions can be placed anywhere there are a few places where they would actually just do more harm than good (principle 2). One example is as a parameter default (define a($b = notify {hi:}) {}).DECISION FOURThe LHS of a resource *instantiation* expression can be an expressionWhat?!? This means you can do:$a = notify$a { hi: }Once again, in clearing up odd cases in the grammar this is opened up to us. This is a very powerful feature to have available. Since this is very useful and fits well into the grammar I don't see this being a temporary thing that would then have to go away later (principle 1).
Why is this? It seems to make the language a lot more powerful, but a lot more complex and harder to read. I’d want to make sure there’s a good reason for that, especially since it takes the core statement in the language and enables it to look and act very differently.
DECISION FIVE (how many of these are there?)A resource with a title of default provides the default parameter values for other resources in the same instantiation expression.Thanks to David Schmitt for this idea!Since we aren't going to change the behavior of resource default expressions (Notify { ... }) it seems like there needs to be something done to provide a better, safer way of specifying defaults. This will allow:notify {default: message => hi;bye: }The result will be a resource of type Notify with title bye and message hi. It is highly unlikely that this will go away (principle 1) as it is syntactic sugar for specifying the parameters for every resource.
I agree, this is cute and simple.
DECISION SIXThere will be a splat operator for resource instantiation expressionsTo make the default resources (decision five) really useful there needs to be a way to reuse the same values across multiple defaults. The current, dangerous, semantics of resource default expressions skirt this issue by making defaults part of the (dynamic) evaluation scope. In order to make the default resources nearly as useful but much safer, we need to add a way to allow reuse of defaults across multiple resource instantiation expressions explicitly (principle 2).$owner_mode = { owner => andy, mode => '777' } # gotta make it securefile { default: *=> $owner_mode;'/home/andy/.bashrc': ;'/home/andy/.ssh/id_rsa': ;}file { '/etc/passwd': *=> $owner_mode }As a side note, do you see what can now be done?$a = notify$b = hi$c = { message => bye }$a { $b: *=> $c }
As above, this makes the language much more powerful and complex, and it seems like makes it possible to build truly unreadable code without trying very hard.
What’s the use case for this, and why is that use case important enough to justify this increase in complexity?
DECISION SEVENundef is not allowed as a titleNot much to say here. notify { undef: } fails (or anything that evaluates to undef)DECISION EIGHTAn array as a title expands to individual resource instantiation expressions with titles of the elements of the array.This isn't really too far off from the current semantics, no real change here. It is only to call out that we are formalizing that as the semantics. An empty array ends up being a noop (no resources instantiated). An array that contains undef will produce an error (see decision seven). The value default can be an element of the array and will produce the default section for the resources being instantiated (as pointless as that seems since they will all have the same body).DECISION NINEDecisions two through eight do not apply to resource default or resource override expressions.Just to make it clear that decision one still holds.CONCLUSIONI think that covers it all. This will be reflected by a revert to some code, modifying the grammar, adding some new evaluation capabilities, including tests, and updating the specification. All of this is falling under PUP-501, PUP-511, and PUP-2898 in some way shape or form.This email was to record the decisions; make them public; double check that Henrik, Joshua and I all had the same understanding of them; and give another chance to everyone to weigh in.I did have one question that I uncovered as I was writing this up. Some feedback on that would be great as well.--Andrew ParkerFreenode: zaphod42Twitter: @aparker42Software DeveloperRegister by May 30th to take advantage of the Early Adopter discount —save $349!
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANhgQXu3HVrWJrTnMgYvbY6%3DR8B%3DvVgts2Uqmwjtj6eJRJsH7g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
John Bollinger
Jul 28, 2014, 9:43:21 AM7/28/14
to puppe...@googlegroups.com
On Friday, July 25, 2014 5:15:30 PM UTC-5, henrik lindberg wrote:
We reasoned that we already have create_resources in frequent use to
solve real issues, so it is needed. I don't think create_resources is
used simply because you can. Instead, it is because you do not
statically know the type, or you want to decide which attributes to set
based on calculations you made. Both of those require create_resources
function. Now it moves into the language.
For what it's worth, every use of create_resources() that I have ever come across or thought about has involved a resource type that is statically known. The usual reason for create_resources() is that the identities and properties of the resources need to be dynamically determined. That is, if I need or want to resort to create_resources() then it's usually because I don't know statically which specific resources (of a known type) need to be managed. Sometimes I also don't know which properties need to be managed or maybe what their target values should be, but that's secondary.
>
> "*=>" looks weird, even for Puppet. What about contracting it to "*>",
> parallel to the plussignment operator (+>) and to the ordinary value
> binding operator (=>)?
>
The rationale is that you normally have:
name => value
Now, you do not have names, and * is often used as a wildcard/splat/
"split it up", implied, the name is picked from the hash, and you may
want to line them up:
file { default:
* => value ;
foo:
mode => '0777' ;
}
Does that make it feel better?
Somewhat, yes. I at least get the mnemonic angle, but it still seems weird.
Is this one operator "*=>" or two "*" "=>"? If one, then are you really going to permit splitting the operator into two tokens? If two, then is this splat operator the same one you described elsewhere? And what is the splat's operand in this context?
With this new, highly expressional approach, it seems like there should be a way to do without the *=> altogether. I mean, if you have a property name / value hash, then why shouldn't you be able to plunk it down undecorated inside a resource instantiation expression? E.g.
file {
default:
$file_defaults;
foo:
mode => '0777';
}
or
file {
default:
$general_file_defaults,
owner => 'ted';
foo:
mode => '0777';
}
or even
file {
default:
$general_file_defaults,
$mymodule_file_defaults,
owner => 'ted';
foo:
mode => '0777';
}
The first proposal was:
title: (expr)
Then, just use the splat operator:
title: *expr
But then it became weird, because it is a special kind of splat that
applies to the entire expression - that felt more confusing.
But haven't you still got a special kind of splat? I think all you've done is alter the spelling to distinguish it from the other splat -- which is better than not -- but it's still kinda confusing.
>
> DECISION EIGHT
>
> An array as a title expands to individual resource instantiation
> expressions with titles of the elements of the array.
>
> This isn't really too far off from the current semantics, no real
> change here.
>
>
>
> Is there any change /at all/ there with respect to what current Puppet
> already supports? (I don't see any, but you've spooked me.)
>
Current implementation silently skips undef entries, balks at an undef
title (not in an array), but allows other types than Strings as
resulting individual titles (with the weird stringification taking place
that we talked about above). Hence, "not too far off". Feel better now?
Yes, thank you.
John
Henrik Lindberg
Jul 28, 2014, 10:33:35 AM7/28/14
to puppe...@googlegroups.com
On 2014-28-07 8:34, Luke Kanies wrote:
> On Jul 24, 2014, at 5:32 PM, Andy Parker <an...@puppetlabs.com
> On Jul 24, 2014, at 5:32 PM, Andy Parker <an...@puppetlabs.com
where it previously was relaxed and/or had undefined behavior.
templates, or calling the create_resources function. It is mostly done
by module authors that integrate with other modules, and where it is not
known at authoring time what the name of the type is.
It allows allows a transformation of data to resources to be expressed
in the puppet language. Again, the alternative is to transform to a
generic hash and then call create_resources. Sometimes people create
their own variants of create_resources as custom function because it
has been difficult to perform such transformations in the puppet language.
We believe that the ability to have indirection for the type is a better
solution than the alternatives, even if looks more complex, it makes the
logic visible and reduces the dependency on Ruby.
Andy Parker
Jul 29, 2014, 2:23:09 PM7/29/14
to puppe...@googlegroups.com
On Thu, Jul 24, 2014 at 5:32 PM, Andy Parker <an...@puppetlabs.com> wrote:
Howdy,Henrik, David, Erik, John, and others have been having some pretty epic conversations around resource expressions, precedence, order of evaluation, and several other topics. What kicked all of that off was us looking for some feedback on decisions we were making for the Puppet 4 language about how resource overrides, defaults, and so on actually work (or don't in some cases). I think we've finally reached some decisions!Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#Lots of information in there, so here is the summary.The principles behind the decisions:1. Only make changes that have a high likelihood of *not* needing to be backed out later.2. Strict or explicit is better than lax or implicit. It uncovers issues and keeps you from lying to yourself.3. Puppet 3 has already stacked up a lot of changes. Do not break manifests unless we really have to.4. Let's not break manifests and then have to break them in almost the same way once we start working on a new catalog system.There are three kinds of resource expression that we have to deal with:1. Resource instantiation2. Resource defaults3. Resource overridesLooking forward, I think it is highly likely that the catalog system that we'll be working on during puppet 4 will be some sort of production (rules) system. In that kind of a world, resource instantiation likely remains as is, but defaults and overrides will end up having to change quite a bit, if not in syntax, at least in semantics.
Replying to myself...tsk tsk. Oh well.
This is to provide an update on what's going on. Henrik put together a bunch of changes to the implementation to carry out these decisions. I worked on the specification to get it to align with these decisions
The change to the specification happened in this PR https://github.com/puppetlabs/puppet-specifications/pull/14
We've already identified one problem in the specification, it doesn't allow nested arrays of strings in the title. I have a note to fix that.
The implementation changes are currently in this PR https://github.com/puppetlabs/puppet/pull/2914
I'm working through adding tests based on the specification and to explore what kinds of changes to the current semantics we have got.
Joshua and I have been working on putting together some tests to not only test the implementation against the specification, but also to see how different it really ends up being from the old implementation. The tests check several different scenarios, but basically, the first item is used as the title of a resource, the second is whether we expect to create a resource or get an error. The third element is either the regexp for the error message or the titles of the resources that actually get created.
This is for the current system:
["thing" ,"resource", ["thing"]],
["[thing]" ,"resource", ["thing"]],
["[[nested, array]]" ,"resource", ["nested", "array"]],
["1" ,"resource", ["1"]], # deprecate?
["3.0" ,"resource", ["3.0"]], # deprecate?
["[1]" ,"resource", ["1"]], # deprecate?
["[3.0]" ,"resource", ["3.0"]], # deprecate?
#["true" ,"resource", ["true"]], literal true doesn't parse. true as a variable parses and creates a resource
#["false" ,"resource", ["false"]], similar problems to [false] below. literal false doesn't parse
["[true]" ,"resource", ["true"]], # this makes no sense given the next case
["[false]" ,"error", /No title provided and :notify is not a valid resource reference/], # *sigh*
#["undef" ,"resource", ["undef"]], # works for variable value, not for literal. deprecate?
["[undef]" ,"resource", ["undef"]], # deprecate?
#["{nested => hash}" ,"resource", ["{nested => hash}"]], doesn't work for a literal hash (syntax error). In a variable it is created, but isn't possible to reference the resource. deprecate?
#["[{nested => hash}]" ,"resource", ["{nested => hash}"]], works as both literal and variable, but isn't possible to reference the resource. deprecate?
["/regexp/" ,"error", /Syntax error/],
["[/regexp/]" ,"error", /Syntax error/],
["default" ,"error", /Syntax error/],
["[default]" ,"error", /Syntax error/],
The current system has a lot of odd outcomes in it. Here are the tests for the future parser (as changed by the PR)
["thing" ,"resource", ["thing"]],
["[thing]" ,"resource", ["thing"]],
["[[nested, array]]" ,"resource", ["nested", "array"]],
["1" ,"error", /Illegal title type.*Expected String, got Integer/],
["3.0" ,"error", /Illegal title type.*Expected String, got Float/],
["[1]" ,"error", /Illegal title type.*Expected String, got Integer/],
["[3.0]" ,"error", /Illegal title type.*Expected String, got Float/],
["true" ,"error", /Illegal title type.*Expected String, got Boolean/],
["false" ,"error", /Illegal title type.*Expected String, got Boolean/],
["[true]" ,"error", /Illegal title type.*Expected String, got Boolean/],
["[false]" ,"error", /Illegal title type.*Expected String, got Boolean/],
["undef" ,"error", /Missing title.*undef/],
["[undef]" ,"error", /Missing title.*undef/],
["{nested => hash}" ,"error", /Illegal title type.*Expected String, got Hash/],
["[{nested => hash}]" ,"error", /Illegal title type.*Expected String, got Hash/],
["/regexp/" ,"error", /Illegal title type.*Expected String, got Regexp/],
["[/regexp/]" ,"error", /Illegal title type.*Expected String, got Regexp/],
["default" ,"resource", []], # nothing created because this is just a local default
["[default]" ,"resource", []],
I'm a little worried that these changes might be too extreme. Thoughts?
The specification still needs a lot of work. It is really hard to piece together right now, but we are working on changing how it is laid out.
Reid Vandewiele
Jul 29, 2014, 2:54:24 PM7/29/14
to puppe...@googlegroups.com
Definitely excited to see this stuff moving forwards.
On Thursday, July 24, 2014 5:32:13 PM UTC-7, Andy Parker wrote:
Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#
According to that doc, syntactically we are transitioning from this:
* Instantiation notify { hi: message => 'hello' }
* Default Notify { message => 'greetings' }
This mostly makes sense except for the use of the literal "default" in the Override syntax. The term "default" implies semantics which I don't think are correct. When using an override it's often the case that previously set values are being explicitly swapped out for different ones. It seems like it would make more sense to change the placeholder word to something that reflects that values set take precedence. E.g.
* Instantiation notify { hi: message => 'hello' }
* Default Notify { message => 'greetings' }
* Override Notify[hi] { message => 'hello there' }
To this:
To this:
* Instantiation notify { hi: message => 'hello' }
* Default Notify { default: message => 'greetings' }
* Default Notify { default: message => 'greetings' }
* Override Notify[hi] { default: message => 'hello there' }
* Override Notify[hi] { override: message => 'hello there' }
Is it an intentional design decision to continue to use the word "default" in the new syntax for resource overrides?
~Reid
Andy Parker
Jul 29, 2014, 3:13:42 PM7/29/14
to puppe...@googlegroups.com
On Tue, Jul 29, 2014 at 11:54 AM, Reid Vandewiele <re...@puppetlabs.com> wrote:
Definitely excited to see this stuff moving forwards.According to that doc, syntactically we are transitioning from this:On Thursday, July 24, 2014 5:32:13 PM UTC-7, Andy Parker wrote:Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#
* Instantiation notify { hi: message => 'hello' }
* Default Notify { message => 'greetings' }* Override Notify[hi] { message => 'hello there' }
To this:* Instantiation notify { hi: message => 'hello' }
* Default Notify { default: message => 'greetings' }* Override Notify[hi] { default: message => 'hello there' }
That was the proposal at one point in time. Lower down in the document you can read through the various proposals for changes that Henrik presented and then the email thread here is the final decisions that we reached. The puppet-specification[1] should contain the language definition of what we will be shipping as the future parser in 3.7 and the real thing in 4.0.
Specifically the changes to Default and Override expressions aren't being done. The problems with the proposal was that it created too many different ways of doing something and we didn't think that the changes would be something that we could even keep as the catalog system evolves. The specific problem we are keeping in mind is the one of "duplicate resources", and the solution that we have in mind makes override and default expressions fit very poorly. Instead we need precedence rules in order to determine which resource expression's value is the one that should be used.
This mostly makes sense except for the use of the literal "default" in the Override syntax. The term "default" implies semantics which I don't think are correct. When using an override it's often the case that previously set values are being explicitly swapped out for different ones. It seems like it would make more sense to change the placeholder word to something that reflects that values set take precedence. E.g.* Override Notify[hi] { override: message => 'hello there' }Is it an intentional design decision to continue to use the word "default" in the new syntax for resource overrides?
~Reid--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/1d192839-f8e3-4d0f-a5f8-1c790e96fca0%40googlegroups.com.
Jason Antman
Jul 30, 2014, 8:23:12 AM7/30/14
to puppe...@googlegroups.com
FWIW,
On Thu, Jul 24, 2014 at 8:32 PM, Andy Parker <an...@puppetlabs.com> wrote:
DECISION TWOResource instantiations are value producing expressionsThe expression based grammar that puppet 4 will be based on changed almost everything into a general expression, which allowed a lot of composition that wasn't possible before. This didn't change resource expressions. Resource expressions could not be assigned ($a = notify {hi:}). That is being changed. This removes several odd corners in the grammar and makes it all more consistent. It is also highly unlikely that it would be removed later (principle 1). The value of a resource expression is a reference to the created resource, or an array of references if there is more than one.
Very very cool.
QUESTION: should the value always be an array of references? That would make it much more predictable.
I'm split on this. On one hand, for people who aren't really comfortable yet with the new language features, it would be more straightforward to have the result be the same as the input (title) type - i.e. if you didn't pass it an array (and don't want to deal with iteration) you don't have to. On the other hand, aside from truthy/falsy differences, I really don't like it when dynamically-typed languages take advantage of that to return different types like this - why make the user type check if they don't have to. "This always returns an array" is easy to work with, especially if the input might be a string or might be an array.
DECISION FOURThe LHS of a resource *instantiation* expression can be an expressionWhat?!? This means you can do:$a = notify$a { hi: }Once again, in clearing up odd cases in the grammar this is opened up to us. This is a very powerful feature to have available. Since this is very useful and fits well into the grammar I don't see this being a temporary thing that would then have to go away later (principle 1).
Agreed with Luke on this one. This scares me, and looks... not like Puppet. I also can't think of a time that I was ever writing a manifest and thought, "gee, I'd really like to make this a package or a service, but sent either one the same params." (ok, I suppose I can see a use case where I have 2 classes in my module that do more or less the same thing and take the same params and I just want one or the other...)
As a side note, do you see what can now be done?$a = notify$b = hi$c = { message => bye }$a { $b: *=> $c }
If I ever get a code review with "$a { $b: *=> $c }" in a .pp file, I'm going to fire the submitter. In all seriousness though, having seen manifests written by some people new to puppet (or looking back 3-4 years in git history) I have some concerns about changes that could have such a negative impact on language readability. The above ensures me that, at some point in my career, I will see:
$a {$foo::bar::baz: *=> $blam::blarg }
Overall, very good work, and I can't wait to see how all this shakes out. I know I've been a bit of a naysayer re: the language improvements. I suppose if lint supports them properly and there are good docs on what not to do, it's no big deal. My worries mainly revolve around (up to now) the language's rigidity, which may have limited many of us, but also prevented less experienced folks from doing anything *too* ugly/bad/unreadable.
-Jason
John Bollinger
Jul 30, 2014, 4:02:27 PM7/30/14
to puppe...@googlegroups.com
On Tuesday, July 29, 2014 1:23:09 PM UTC-5, Andy Parker wrote:
On Thu, Jul 24, 2014 at 5:32 PM, Andy Parker <an...@puppetlabs.com> wrote:
Howdy,Henrik, David, Erik, John, and others have been having some pretty epic conversations around resource expressions, precedence, order of evaluation, and several other topics. What kicked all of that off was us looking for some feedback on decisions we were making for the Puppet 4 language about how resource overrides, defaults, and so on actually work (or don't in some cases). I think we've finally reached some decisions!Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#Lots of information in there, so here is the summary.The principles behind the decisions:1. Only make changes that have a high likelihood of *not* needing to be backed out later.2. Strict or explicit is better than lax or implicit. It uncovers issues and keeps you from lying to yourself.3. Puppet 3 has already stacked up a lot of changes. Do not break manifests unless we really have to.4. Let's not break manifests and then have to break them in almost the same way once we start working on a new catalog system.There are three kinds of resource expression that we have to deal with:1. Resource instantiation2. Resource defaults3. Resource overridesLooking forward, I think it is highly likely that the catalog system that we'll be working on during puppet 4 will be some sort of production (rules) system. In that kind of a world, resource instantiation likely remains as is, but defaults and overrides will end up having to change quite a bit, if not in syntax, at least in semantics.Replying to myself...tsk tsk. Oh well.This is to provide an update on what's going on. Henrik put together a bunch of changes to the implementation to carry out these decisions. I worked on the specification to get it to align with these decisionsThe change to the specification happened in this PR https://github.com/puppetlabs/puppet-specifications/pull/14We've already identified one problem in the specification, it doesn't allow nested arrays of strings in the title. I have a note to fix that.
Is support for nested string arrays an intentional current feature? I hadn't thought so, but maybe you slipped in automatic flattening of array titles sometime when I wasn't looking. If it's not a current feature, then does it need to be a future feature?
The implementation changes are currently in this PR https://github.com/puppetlabs/puppet/pull/2914I'm working through adding tests based on the specification and to explore what kinds of changes to the current semantics we have got.Joshua and I have been working on putting together some tests to not only test the implementation against the specification, but also to see how different it really ends up being from the old implementation. The tests check several different scenarios, but basically, the first item is used as the title of a resource, the second is whether we expect to create a resource or get an error. The third element is either the regexp for the error message or the titles of the resources that actually get created.This is for the current system:["thing" ,"resource", ["thing"]],["[thing]" ,"resource", ["thing"]],["[[nested, array]]" ,"resource", ["nested", "array"]],
And that last one actually works? I would have expected this:
["[[nested, array]]" ,"resource", ["nestedarray"]]
, which I don't categorize as "working".
["1" ,"resource", ["1"]], # deprecate?["3.0" ,"resource", ["3.0"]], # deprecate?["[1]" ,"resource", ["1"]], # deprecate?["[3.0]" ,"resource", ["3.0"]], # deprecate?#["true" ,"resource", ["true"]], literal true doesn't parse. true as a variable parses and creates a resource#["false" ,"resource", ["false"]], similar problems to [false] below. literal false doesn't parse["[true]" ,"resource", ["true"]], # this makes no sense given the next case["[false]" ,"error", /No title provided and :notify is not a valid resource reference/], # *sigh*#["undef" ,"resource", ["undef"]], # works for variable value, not for literal. deprecate?["[undef]" ,"resource", ["undef"]], # deprecate?
I'd say of the above that any value that works as a title when specified indirectly via a variable should also work when specified as a literal, and vise versa. I would be ok with requiring actual strings instead of stringifying values of other types. I believe Henrik promoted that approach.
#["{nested => hash}" ,"resource", ["{nested => hash}"]], doesn't work for a literal hash (syntax error). In a variable it is created, but isn't possible to reference the resource. deprecate?#["[{nested => hash}]" ,"resource", ["{nested => hash}"]], works as both literal and variable, but isn't possible to reference the resource. deprecate?
You can't reference the literal because the resource name is not the hash, it is the stringification of the hash. IIRC works out to "nestedhash" in this case. I would be all for deprecating that behavior. As far as I know, it is never used except by mistake, and the errors resulting from that mistake are confusing to the casual Puppet user.
["/regexp/" ,"error", /Syntax error/],["[/regexp/]" ,"error", /Syntax error/],["default" ,"error", /Syntax error/],["[default]" ,"error", /Syntax error/],The current system has a lot of odd outcomes in it.
I'd agree with that, but odd as they are, they are also relatively consistent if you look at it from the right perspective. Mostly the behavior springs from the fact that Puppet expects resource titles to be strings, and from the fact that the grammar recognizes and distinguishes the data types of many productions. There's a bit of a tension there, though, because if you sneak a non-string individual resource title to the back end by, say, hiding it in a variable, then the back end just stringifies it instead of rejecting it.
Here are the tests for the future parser (as changed by the PR)["thing" ,"resource", ["thing"]],["[thing]" ,"resource", ["thing"]],["[[nested, array]]" ,"resource", ["nested", "array"]],
I still find that last one surprising, but if it really matches the current behavior then so be it. What's the rule here, though? Array titles are automatically flattened?
["1" ,"error", /Illegal title type.*Expected String, got Integer/],["3.0" ,"error", /Illegal title type.*Expected String, got Float/],["[1]" ,"error", /Illegal title type.*Expected String, got Integer/],["[3.0]" ,"error", /Illegal title type.*Expected String, got Float/],["true" ,"error", /Illegal title type.*Expected String, got Boolean/],["false" ,"error", /Illegal title type.*Expected String, got Boolean/],["[true]" ,"error", /Illegal title type.*Expected String, got Boolean/],["[false]" ,"error", /Illegal title type.*Expected String, got Boolean/],["undef" ,"error", /Missing title.*undef/],["[undef]" ,"error", /Missing title.*undef/],["{nested => hash}" ,"error", /Illegal title type.*Expected String, got Hash/],["[{nested => hash}]" ,"error", /Illegal title type.*Expected String, got Hash/],["/regexp/" ,"error", /Illegal title type.*Expected String, got Regexp/],["[/regexp/]" ,"error", /Illegal title type.*Expected String, got Regexp/],["default" ,"resource", []], # nothing created because this is just a local default["[default]" ,"resource", []],I'm a little worried that these changes might be too extreme. Thoughts?
Beyond what I've already said, I don't personally have a problem with any of the changes highlighted here.
John
Andy Parker
Jul 30, 2014, 5:21:37 PM7/30/14
to puppe...@googlegroups.com
On Wed, Jul 30, 2014 at 1:02 PM, John Bollinger <john.bo...@stjude.org> wrote:
On Tuesday, July 29, 2014 1:23:09 PM UTC-5, Andy Parker wrote:On Thu, Jul 24, 2014 at 5:32 PM, Andy Parker <an...@puppetlabs.com> wrote:
Howdy,Henrik, David, Erik, John, and others have been having some pretty epic conversations around resource expressions, precedence, order of evaluation, and several other topics. What kicked all of that off was us looking for some feedback on decisions we were making for the Puppet 4 language about how resource overrides, defaults, and so on actually work (or don't in some cases). I think we've finally reached some decisions!Henrik took all of the ideas and started trying to work out what we could do and what we couldn't. Those are in a writeup at https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit#Lots of information in there, so here is the summary.The principles behind the decisions:1. Only make changes that have a high likelihood of *not* needing to be backed out later.2. Strict or explicit is better than lax or implicit. It uncovers issues and keeps you from lying to yourself.3. Puppet 3 has already stacked up a lot of changes. Do not break manifests unless we really have to.4. Let's not break manifests and then have to break them in almost the same way once we start working on a new catalog system.There are three kinds of resource expression that we have to deal with:1. Resource instantiation2. Resource defaults3. Resource overridesLooking forward, I think it is highly likely that the catalog system that we'll be working on during puppet 4 will be some sort of production (rules) system. In that kind of a world, resource instantiation likely remains as is, but defaults and overrides will end up having to change quite a bit, if not in syntax, at least in semantics.Replying to myself...tsk tsk. Oh well.This is to provide an update on what's going on. Henrik put together a bunch of changes to the implementation to carry out these decisions. I worked on the specification to get it to align with these decisionsThe change to the specification happened in this PR https://github.com/puppetlabs/puppet-specifications/pull/14We've already identified one problem in the specification, it doesn't allow nested arrays of strings in the title. I have a note to fix that.
Is support for nested string arrays an intentional current feature? I hadn't thought so, but maybe you slipped in automatic flattening of array titles sometime when I wasn't looking. If it's not a current feature, then does it need to be a future feature?
As a note: I've updated the tests to be much more explicit and cover more of the functionality: https://github.com/hlindberg/puppet/blob/PUP-2898_resource-expressions/spec/integration/parser/resource_expressions_spec.rb There are still some cases missing. Hailee (new developer at PL on the team. She was an intern a couple times and worked on facter), is tackling one bug and was going to check the tests against the specification and fill in any gaps that I've missed.
Allowing arrays of strings was definitely intentional (https://github.com/puppetlabs/puppet/blob/master/lib/puppet/parser/ast/resource.rb#L42). If the consequence of allowing nested arrays of strings was intentional is harder to say. The tests for that code don't check for nested arrays and the docs for resources don't mention nested arrays. However, it does work:
> be puppet apply -e 'notify { [[a, b]]: }'
Notice: Compiled catalog for aparker.corp.puppetlabs.net in environment production in 0.02 seconds
Notice: b
Notice: /Stage[main]/Main/Notify[b]/message: defined 'message' as 'b'
Notice: a
Notice: /Stage[main]/Main/Notify[a]/message: defined 'message' as 'a'
Notice: Finished catalog run in 0.06 seconds
The reasoning given by Henrik for why this should stay around is because you couldn't previously directly concatenate two arrays in the puppet language leading to people possibly doing this, either intentionally or unintentionally:
$a = [a, b]
$b = [c, d]
notify { [$a, $b]: }
And of course this kind of a situation can be setup somewhere along a long chain of constructing titles and passing things around, so it is really hard to tell if it is really used or not. I did a check through a large body of puppet code (a dump of the forge modules) and I haven't found anything as explicit as what I have above, but every time I think something isn't done it turns out that there is someone who was doing it.
Just to note, that can now be done as:
$a = [a, b]
$b = [c, d]
notify { $a + $b: }
Any thoughts on how disruptive it might be to remove this? Or whether this feature is even bad to keep around? It seems pretty harmless to me to just keep it.
The implementation changes are currently in this PR https://github.com/puppetlabs/puppet/pull/2914I'm working through adding tests based on the specification and to explore what kinds of changes to the current semantics we have got.Joshua and I have been working on putting together some tests to not only test the implementation against the specification, but also to see how different it really ends up being from the old implementation. The tests check several different scenarios, but basically, the first item is used as the title of a resource, the second is whether we expect to create a resource or get an error. The third element is either the regexp for the error message or the titles of the resources that actually get created.This is for the current system:["thing" ,"resource", ["thing"]],["[thing]" ,"resource", ["thing"]],["[[nested, array]]" ,"resource", ["nested", "array"]],
And that last one actually works? I would have expected this:
["[[nested, array]]" ,"resource", ["nestedarray"]]
, which I don't categorize as "working".
Yep. That one creates two resources with those two titles. I'm willing to change it and be stricter, but I think it comes down to making an argument for why allowing nested arrays is actually *bad*.
["1" ,"resource", ["1"]], # deprecate?["3.0" ,"resource", ["3.0"]], # deprecate?["[1]" ,"resource", ["1"]], # deprecate?["[3.0]" ,"resource", ["3.0"]], # deprecate?#["true" ,"resource", ["true"]], literal true doesn't parse. true as a variable parses and creates a resource#["false" ,"resource", ["false"]], similar problems to [false] below. literal false doesn't parse["[true]" ,"resource", ["true"]], # this makes no sense given the next case["[false]" ,"error", /No title provided and :notify is not a valid resource reference/], # *sigh*#["undef" ,"resource", ["undef"]], # works for variable value, not for literal. deprecate?["[undef]" ,"resource", ["undef"]], # deprecate?
I'd say of the above that any value that works as a title when specified indirectly via a variable should also work when specified as a literal, and vise versa. I would be ok with requiring actual strings instead of stringifying values of other types. I believe Henrik promoted that approach.
You are right. My "deprecate?" comments above where me wondering if the current parser should issue warnings. Now that I think about it, issuing warnings in these situations is impossible because the current parser doesn't make a clear distinction between numbers and strings. It is probably better to just rely on the new one failing when it encounters these situations.
#["{nested => hash}" ,"resource", ["{nested => hash}"]], doesn't work for a literal hash (syntax error). In a variable it is created, but isn't possible to reference the resource. deprecate?#["[{nested => hash}]" ,"resource", ["{nested => hash}"]], works as both literal and variable, but isn't possible to reference the resource. deprecate?
You can't reference the literal because the resource name is not the hash, it is the stringification of the hash. IIRC works out to "nestedhash" in this case. I would be all for deprecating that behavior. As far as I know, it is never used except by mistake, and the errors resulting from that mistake are confusing to the casual Puppet user.
It works:
> be puppet apply -e '$a = { my => hash } notify { $a: } notify { second: require => Notify[{"my"=>"hash"}] }'
Notice: Compiled catalog for aparker.corp.puppetlabs.net in environment production in 0.02 seconds
Notice: {"my"=>"hash"}
Notice: /Stage[main]/Main/Notify[{"my"=>"hash"}]/message: defined 'message' as '{"my"=>"hash"}'
Notice: second
Notice: /Stage[main]/Main/Notify[second]/message: defined 'message' as 'second'
Notice: Finished catalog run in 0.06 seconds
And it works in a surprising number of situations. The title of the resource ends up being the hash:
{
"type": "Notify",
"title": {
"my": "hash"
},
"tags": ["notify","class"],
"file": "/Users/andy/work/puppet/t.pp",
"line": 2,
"exported": false,
"parameters": {
"name": "{\"my\"=>\"hash\"}"
}
}
The name parameter ends up being a stringification of the hash and edges it is also a stringification:
{
"source": "Class[main]",
"target": "Notify[{\"my\"=>\"hash\"}]"
}
This can sometimes go wrong (https://tickets.puppetlabs.com/browse/PUP-2935 <- I haven't figured out what happened there other than using a hash as a title going awry).
My test might have just been trying to find the resource incorrectly...
["/regexp/" ,"error", /Syntax error/],["[/regexp/]" ,"error", /Syntax error/],["default" ,"error", /Syntax error/],["[default]" ,"error", /Syntax error/],The current system has a lot of odd outcomes in it.
I'd agree with that, but odd as they are, they are also relatively consistent if you look at it from the right perspective. Mostly the behavior springs from the fact that Puppet expects resource titles to be strings, and from the fact that the grammar recognizes and distinguishes the data types of many productions. There's a bit of a tension there, though, because if you sneak a non-string individual resource title to the back end by, say, hiding it in a variable, then the back end just stringifies it instead of rejecting it.
Exactly. And that is why I'm a little nervous about just disallowing all of these types. However, it will now fail fast and so the migration path to the new system is: get rid of warnings, run with --parser future, fix errors, check that your systems still work. OTOH what do we lose from just continuing to stringify certain data types?
Here are the tests for the future parser (as changed by the PR)["thing" ,"resource", ["thing"]],["[thing]" ,"resource", ["thing"]],["[[nested, array]]" ,"resource", ["nested", "array"]],
I still find that last one surprising, but if it really matches the current behavior then so be it. What's the rule here, though? Array titles are automatically flattened?
Right. Array titles are flattened and the flattened form is checked for duplicates.
["1" ,"error", /Illegal title type.*Expected String, got Integer/],["3.0" ,"error", /Illegal title type.*Expected String, got Float/],["[1]" ,"error", /Illegal title type.*Expected String, got Integer/],["[3.0]" ,"error", /Illegal title type.*Expected String, got Float/],["true" ,"error", /Illegal title type.*Expected String, got Boolean/],["false" ,"error", /Illegal title type.*Expected String, got Boolean/],["[true]" ,"error", /Illegal title type.*Expected String, got Boolean/],["[false]" ,"error", /Illegal title type.*Expected String, got Boolean/],["undef" ,"error", /Missing title.*undef/],["[undef]" ,"error", /Missing title.*undef/],["{nested => hash}" ,"error", /Illegal title type.*Expected String, got Hash/],["[{nested => hash}]" ,"error", /Illegal title type.*Expected String, got Hash/],["/regexp/" ,"error", /Illegal title type.*Expected String, got Regexp/],["[/regexp/]" ,"error", /Illegal title type.*Expected String, got Regexp/],["default" ,"resource", []], # nothing created because this is just a local default["[default]" ,"resource", []],I'm a little worried that these changes might be too extreme. Thoughts?
Beyond what I've already said, I don't personally have a problem with any of the changes highlighted here.
John
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/76d12bc2-5917-42b9-b397-c5773df8b9fe%40googlegroups.com.
Henrik Lindberg
Jul 30, 2014, 6:57:14 PM7/30/14
to puppe...@googlegroups.com
On 2014-30-07 23:21, Andy Parker wrote:
> Exactly. And that is why I'm a little nervous about just disallowing all
> of these types. However, it will now fail fast and so the migration path
> to the new system is: get rid of warnings, run with --parser future, fix
> errors, check that your systems still work. OTOH what do we lose from
> just continuing to stringify certain data types?
You lose control :-)
> Exactly. And that is why I'm a little nervous about just disallowing all
> of these types. However, it will now fail fast and so the migration path
> to the new system is: get rid of warnings, run with --parser future, fix
> errors, check that your systems still work. OTOH what do we lose from
> just continuing to stringify certain data types?
More specifically you loose control over how Numbers are stringified. Do
you want octal, hex or decimal? And if you are presenting a collection,
should all nested numbers be octal, hex, or decimal?
(The default stringification uses decimal).
Do you expect to be able to transform the string back to a data
structure? (It is not possible since undefs stringify as empty strings,
and there are no delimiters around String values.)
Also note that 4x stringification does not guarantee that it produces an
identical result to 3x, since 3x just used Ruby to_s on what it got,
where 4x uses the rules for string interpolation.
So, it was very deliberate to only allow string titles. It also removes
the ambiguity of the semantics of nested collections (i.e. it is safe to
flatten since you cannot possibly mean a stringified array).
If someone really wants a title that is a stringified hash they can
interpolate the value into a string.
John Bollinger
Jul 31, 2014, 4:16:35 PM7/31/14
to puppe...@googlegroups.com
Surprising. Leaving aside the tests for a moment, let's consider what the docs say:
1. They specifically say that resource titles are strings (http://docs.puppetlabs.com/puppet/3/reference/lang_resources.html#syntax): "The title, which is a string"; "The title is an identifying string."
2. They say you can specify an array of strings as a resource title, as shorthand for multiple resource declarations (http://docs.puppetlabs.com/puppet/3/reference/lang_resources.html#array-of-titles). This one is a little more open, but at best Puppet's behavior for titles consisting of arrays containing non-string elements (including other arrays) is undocumented.
There are basically two approaches consistent with the docs. Puppet could either stringify non-strings presented as resource titles, or it could reject them. Current Puppet might do either one, depending on the specifics of the situation, and I agree that that inconsistency is a problem.
I can accept the nested array behavior as the result of applying the array title rule recursively, though I don't think that's the most reasonable way to read the docs. If you are determined to keep it then the docs should be clarified.
The reasoning given by Henrik for why this should stay around is because you couldn't previously directly concatenate two arrays in the puppet language leading to people possibly doing this, either intentionally or unintentionally:$a = [a, b]$b = [c, d]notify { [$a, $b]: }And of course this kind of a situation can be setup somewhere along a long chain of constructing titles and passing things around, so it is really hard to tell if it is really used or not. I did a check through a large body of puppet code (a dump of the forge modules) and I haven't found anything as explicit as what I have above, but every time I think something isn't done it turns out that there is someone who was doing it.Just to note, that can now be done as:$a = [a, b]$b = [c, d]notify { $a + $b: }Any thoughts on how disruptive it might be to remove this? Or whether this feature is even bad to keep around? It seems pretty harmless to me to just keep it.
I think it's weird, and I think that although it has certain reasonable uses, it also provides for some surprising manifest development errors. I do not personally rely on the feature, but I cannot say to what extent, if any, others rely on it. I will not object to keeping it around, so long as you document it.
And that last one actually works? I would have expected this:
["[[nested, array]]" ,"resource", ["nestedarray"]]
, which I don't categorize as "working".Yep. That one creates two resources with those two titles. I'm willing to change it and be stricter, but I think it comes down to making an argument for why allowing nested arrays is actually *bad*.
I'm not prepared to present such an argument.
#["{nested => hash}" ,"resource", ["{nested => hash}"]], doesn't work for a literal hash (syntax error). In a variable it is created, but isn't possible to reference the resource. deprecate?#["[{nested => hash}]" ,"resource", ["{nested => hash}"]], works as both literal and variable, but isn't possible to reference the resource. deprecate?
You can't reference the literal because the resource name is not the hash, it is the stringification of the hash. IIRC works out to "nestedhash" in this case. I would be all for deprecating that behavior. As far as I know, it is never used except by mistake, and the errors resulting from that mistake are confusing to the casual Puppet user.
It works:> be puppet apply -e '$a = { my => hash } notify { $a: } notify { second: require => Notify[{"my"=>"hash"}] }'Notice: Compiled catalog for aparker.corp.puppetlabs.net in environment production in 0.02 secondsNotice: {"my"=>"hash"}Notice: /Stage[main]/Main/Notify[{"my"=>"hash"}]/message: defined 'message' as '{"my"=>"hash"}'Notice: secondNotice: /Stage[main]/Main/Notify[second]/message: defined 'message' as 'second'Notice: Finished catalog run in 0.06 seconds
This is a bug. Per the docs, resource titles are strings.
And it works in a surprising number of situations. The title of the resource ends up being the hash:
It cannot (should not) be a hash. Resource titles are strings.
{"type": "Notify","title": {"my": "hash"},"tags": ["notify","class"],"file": "/Users/andy/work/puppet/t.pp","line": 2,"exported": false,"parameters": {"name": "{\"my\"=>\"hash\"}"}}The name parameter ends up being a stringification of the hash and edges it is also a stringification:{"source": "Class[main]","target": "Notify[{\"my\"=>\"hash\"}]"}This can sometimes go wrong (https://tickets.puppetlabs.com/browse/PUP-2935 <- I haven't figured out what happened there other than using a hash as a title going awry).
Perhaps that issue provides a clue, though, in that it describes different behavior under Ruby 1.8 than under Ruby 1.9. Does your test yield the same result under both versions? It may be that accepting a hash as a title without stringifying it results from a change in semantics between Ruby versions.
Or since there was no test for the behavior, perhaps it was an unintended consequence of some other change.
Exactly. And that is why I'm a little nervous about just disallowing all of these types. However, it will now fail fast and so the migration path to the new system is: get rid of warnings, run with --parser future, fix errors, check that your systems still work. OTOH what do we lose from just continuing to stringify certain data types?
If you stringify only certain data types then you lose consistency, plus you fail to comply with the docs with respect to those that you do not stringify. You also give up an invariant on which otherwise you could rely.
As I already said, I'm prepared to rationalize the behavior for nested arrays as a recursive application of the array-title feature. If you intend to keep that then I would prefer to see it documented that way, as opposed to as implicit array flattening. Considering it recursive involves less magic, even though both descriptions predict the same result.
John
David Schmitt
Aug 1, 2014, 3:39:56 AM8/1/14
to puppe...@googlegroups.com
On 2014-07-31 22:16, John Bollinger wrote:
[good points]
Just a quick note that I'm mainly agreeing with John's points: Automatic
(and inconsistent) stringification of non-string-titles is weird and
confusing, thus should be avoided. Automatic flattening of
arrays-of-strings is weird, but probably not confusing, thus can be
allowed, should be documented and discouraged (e.g. point in the docs to
explicit flattening).
Regards, David
--
* Always looking for people I can help with awesome projects *
G+: https://plus.google.com/+DavidSchmitt
Blog: http://club.black.co.at/log/
LinkedIn: http://at.linkedin.com/in/davidschmitt
[good points]
Just a quick note that I'm mainly agreeing with John's points: Automatic
(and inconsistent) stringification of non-string-titles is weird and
confusing, thus should be avoided. Automatic flattening of
arrays-of-strings is weird, but probably not confusing, thus can be
allowed, should be documented and discouraged (e.g. point in the docs to
explicit flattening).
Regards, David
--
* Always looking for people I can help with awesome projects *
G+: https://plus.google.com/+DavidSchmitt
Blog: http://club.black.co.at/log/
LinkedIn: http://at.linkedin.com/in/davidschmitt
Andy Parker
Aug 1, 2014, 12:28:39 PM8/1/14
to puppe...@googlegroups.com
On Fri, Aug 1, 2014 at 12:39 AM, David Schmitt <da...@dasz.at> wrote:
On 2014-07-31 22:16, John Bollinger wrote:
[good points]
Just a quick note that I'm mainly agreeing with John's points: Automatic (and inconsistent) stringification of non-string-titles is weird and confusing, thus should be avoided. Automatic flattening of arrays-of-strings is weird, but probably not confusing, thus can be allowed, should be documented and discouraged (e.g. point in the docs to explicit flattening).
Alright, I'll take that as consensus then (or at least consent).
The implementation that is on that branch will remain as is, and the spec will be taken as the final form WRT resource expressions.
Regards, David
--
* Always looking for people I can help with awesome projects *
G+: https://plus.google.com/+DavidSchmitt
Blog: http://club.black.co.at/log/
LinkedIn: http://at.linkedin.com/in/davidschmitt
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/53DB4444.9080408%40dasz.at.
Luke Kanies
Aug 3, 2014, 4:10:24 PM8/3/14
to puppe...@googlegroups.com
On Aug 1, 2014, at 9:28 AM, Andy Parker <an...@puppetlabs.com> wrote:
On Fri, Aug 1, 2014 at 12:39 AM, David Schmitt <da...@dasz.at> wrote:On 2014-07-31 22:16, John Bollinger wrote:
[good points]
Just a quick note that I'm mainly agreeing with John's points: Automatic (and inconsistent) stringification of non-string-titles is weird and confusing, thus should be avoided. Automatic flattening of arrays-of-strings is weird, but probably not confusing, thus can be allowed, should be documented and discouraged (e.g. point in the docs to explicit flattening).Alright, I'll take that as consensus then (or at least consent).The implementation that is on that branch will remain as is, and the spec will be taken as the final form WRT resource expressions.
Can you be a little more specific about what ‘that’ is here?
John walked through a bunch of specific cases, and I’d like to make sure we have a summary of the desired behaviors in each of those cases, and whether those behaviors will be changes or just formalization of existing behavior.
Thanks.
Luke Kanies
Aug 3, 2014, 4:18:43 PM8/3/14
to puppe...@googlegroups.com
On Jul 28, 2014, at 6:43 AM, John Bollinger <john.bo...@stjude.org> wrote:
On Friday, July 25, 2014 5:15:30 PM UTC-5, henrik lindberg wrote:
We reasoned that we already have create_resources in frequent use to
solve real issues, so it is needed. I don't think create_resources is
used simply because you can. Instead, it is because you do not
statically know the type, or you want to decide which attributes to set
based on calculations you made. Both of those require create_resources
function. Now it moves into the language.
For what it's worth, every use of create_resources() that I have ever come across or thought about has involved a resource type that is statically known. The usual reason for create_resources() is that the identities and properties of the resources need to be dynamically determined. That is, if I need or want to resort to create_resources() then it's usually because I don't know statically which specific resources (of a known type) need to be managed. Sometimes I also don't know which properties need to be managed or maybe what their target values should be, but that's secondary.
Exactly. This is the more normal case, and is the one we should optimize for. And, IMO, the one that create_resources works just dandily for. I don’t see the need to promote a function to syntax in this case.
Yeah, I’m not at all a fan of the ‘* => foo’ syntax. It seems strange in both the lexer and the parser, and it strikes me as a far large change to the system without a large return.
If we conclude we need significantly more power in the resource parameters, then I’d prefer to do something like formally treat it as a hash, and support some kind of hash operations there. E.g., something like:
file {
default:
$some_defaults + $other_defaults + {owner => ‘ted’};
foo:
mode => ‘0777’
}
This expands how we think about the RHS of a resource declaration, but I think it’s more generically powerful and requires far less specialized syntax.
The first proposal was:
title: (expr)
Then, just use the splat operator:
title: *expr
But then it became weird, because it is a special kind of splat that
applies to the entire expression - that felt more confusing.
But haven't you still got a special kind of splat? I think all you've done is alter the spelling to distinguish it from the other splat -- which is better than not -- but it's still kinda confusing.
Agreed.
Luke Kanies
Aug 3, 2014, 4:22:16 PM8/3/14
to puppe...@googlegroups.com
[…]
I know this wouldn’t cover every case, but wouldn’t it cover the majority of cases, and thus be a better trade-off? That is, we have a somewhat less powerful language, but a much more accessible language, and it comfortably covers the 95%+ use cases while still being usable by 95%+ of its users?
Luke Kanies
Aug 3, 2014, 4:30:19 PM8/3/14
to puppe...@googlegroups.com
Here are the tests for the future parser (as changed by the PR)["thing" ,"resource", ["thing"]],["[thing]" ,"resource", ["thing"]],["[[nested, array]]" ,"resource", ["nested", "array"]],
I still find that last one surprising, but if it really matches the current behavior then so be it. What's the rule here, though? Array titles are automatically flattened?
Just a small note here: I implemented this behavior intentionally, because it seemed the least of the available evils. When I first added the ability for resources to have arrays as titles, I didn’t have any available operations for arrays. So, my choice was: Fail if you used nested arrays; change resources to support arrays as titles or behave appropriately if they got one and couldn’t handle it; add complex array operations (so you could do things like flatten arrays); or just flatten for you.
In most cases, it’s not been a problem, but as we’re looking at building a more coherent and consistent language, it makes sense to revisit this behavior.
Note that when I implemented this, Puppet had almost no functions available by default, and no such thing as calling methods on objects, so implementing “flatten($array)” or “$array.flatten” would have been a pretty big leap.
I think either would be less of a leap today, and it probably makes more sense to ask the user to transform their data into a valid form than to try to do magic transformations.
And, as discovered, we clearly need to become a lot more careful in what kind of data we accept in these fields overall.
So, to summarize: Yes, this is intentional, and it made sense at the time. It probably doesn’t make sense as the default behavior much longer, though.
Henrik Lindberg
Aug 3, 2014, 6:25:43 PM8/3/14
to puppe...@googlegroups.com
On 2014-03-08 22:30, Luke Kanies wrote:
> On Jul 30, 2014, at 1:02 PM, John Bollinger <john..bollinger@stjude.org
While true, it is also a change we do not have to make since it is
perfectly ok to continue to flatten the title array (there is no other
meaningful interpretation). We will break backwards compatibility if we
make it an error.
> On Jul 30, 2014, at 1:02 PM, John Bollinger <john..bollinger@stjude.org
perfectly ok to continue to flatten the title array (there is no other
meaningful interpretation). We will break backwards compatibility if we
make it an error.
Henrik Lindberg
Aug 3, 2014, 6:45:08 PM8/3/14
to puppe...@googlegroups.com
On 2014-03-08 22:18, Luke Kanies wrote:
> On Jul 28, 2014, at 6:43 AM, John Bollinger <john..bollinger@stjude.org
It was trivial to implement. There was nothing difficult about this in
the lexer / parser. The '* =>' is actually two tokens (nothing was added
in the lexer to support this).
> If we conclude we need significantly more power in the resource
> parameters, then I’d prefer to do something like formally treat it as a
> hash, and support some kind of hash operations there. E.g., something like:
>
> file {
> default:
> $some_defaults + $other_defaults + {owner => ‘ted’};
> foo:
> mode => ‘0777’
> }
>
The splat rocket works for all titled resources, not just default.
And it is formalized to accept a hash where the keys must be valid as
names of attributes.
> This expands how we think about the RHS of a resource declaration, but I
> think it’s more generically powerful and requires far less specialized
> syntax.
>
Sorry, but it is actually less powerful, and requires more specialized
handling in the grammar. When using a token that indicates that "here
follows a hash of attribute-name to value mapping", it is much simpler.
Have any other suggestions to replace '* =>' ? Almost anything that is
not already a valid unary operator (i.e. !, *, +, -) or significant
punctuation (, . ; : { } [ ] ( )) could be used (or combination of
tokens, like the now implemented '* =>')
Here are some that have come up:
file { default: => $hsh }
file { default: *> $hsh }
> On Jul 28, 2014, at 6:43 AM, John Bollinger <john..bollinger@stjude.org
> <mailto:john.bo...@stjude.org>> wrote:
>
>>
>>
>> On Friday, July 25, 2014 5:15:30 PM UTC-5, henrik lindberg wrote:
>>
>>
>> We reasoned that we already have create_resources in frequent use to
>> solve real issues, so it is needed. I don't think create_resources is
>> used simply because you can. Instead, it is because you do not
>> statically know the type, or you want to decide which attributes
>> to set
>> based on calculations you made. Both of those require
>> create_resources
>> function. Now it moves into the language.
>>
>>
>>
>> For what it's worth, every use of create_resources() that I have ever
>> come across or thought about has involved a resource type that is
>> statically known. The usual reason for create_resources() is that the
>> *identities* and properties of the resources need to be dynamically
>
>>
>>
>> On Friday, July 25, 2014 5:15:30 PM UTC-5, henrik lindberg wrote:
>>
>>
>> We reasoned that we already have create_resources in frequent use to
>> solve real issues, so it is needed. I don't think create_resources is
>> used simply because you can. Instead, it is because you do not
>> statically know the type, or you want to decide which attributes
>> to set
>> based on calculations you made. Both of those require
>> create_resources
>> function. Now it moves into the language.
>>
>>
>>
>> For what it's worth, every use of create_resources() that I have ever
>> come across or thought about has involved a resource type that is
>> statically known. The usual reason for create_resources() is that the
>> determined. That is, if I need or want to resort to
>> create_resources() then it's /usually/ because I don't know statically
>> which specific resources (of a known type) need to be managed.
>> Sometimes I /also/ don't know which properties need to be managed or
the lexer / parser. The '* =>' is actually two tokens (nothing was added
in the lexer to support this).
> If we conclude we need significantly more power in the resource
> parameters, then I’d prefer to do something like formally treat it as a
> hash, and support some kind of hash operations there. E.g., something like:
>
> file {
> default:
> $some_defaults + $other_defaults + {owner => ‘ted’};
> foo:
> mode => ‘0777’
> }
>
And it is formalized to accept a hash where the keys must be valid as
names of attributes.
> This expands how we think about the RHS of a resource declaration, but I
> think it’s more generically powerful and requires far less specialized
> syntax.
>
handling in the grammar. When using a token that indicates that "here
follows a hash of attribute-name to value mapping", it is much simpler.
Have any other suggestions to replace '* =>' ? Almost anything that is
not already a valid unary operator (i.e. !, *, +, -) or significant
punctuation (, . ; : { } [ ] ( )) could be used (or combination of
tokens, like the now implemented '* =>')
Here are some that have come up:
file { default: => $hsh }
file { default: *> $hsh }
Henrik Lindberg
Aug 3, 2014, 7:32:39 PM8/3/14
to puppe...@googlegroups.com
parser this documentation is not yet produced, so it is a matter of
looking at closed tickets. (Most breaking changes were done prior to 3.7
future parser).
From the top of my mind..
Earlier breaking changes include:
* Numbers are Numeric and must be valid numbers, quote them if they
should be strings.
* An upper cased bare word is always a reference to a type (must be
quoted if you want it to be a string).
* You are no longer allowed to add to arrays and hashes (they are immutable)
* Bugs in operators +, -, *, /, in, >, <, <=, >=, ==, and != could
produce odd results, and if you relied on those quirks the fixes could
be perceived as breaking change.
* =~ and !~ requires the LHS to have String result (it does not
stringify before matching)
* You cannot have a class or defined type named 'class'
* You cannot assign to numeric variables (i.e. $1 = 10)
* Names of classes and defines cannot start with a digit
In 3.7:
* Undef is no longer equal to empty string
* Empty string is truthy
* Attribute names 'false', and 'true' are no longer accepted
> [...]
>>
>> We believe that the ability to have indirection for the type is a better solution than the alternatives, even if looks more complex, it makes the logic visible and reduces the dependency on Ruby.
>
> I am not convinced. Based on what I've seen so far, I prefer transformation to a hash, and using a function. This is a lot of syntactical power, and it is being done to replace what seems like relatively simple hash operations and a simple function.
>
> I know this wouldn't cover every case, but wouldn't it cover the majority of cases, and thus be a better trade-off? That is, we have a somewhat less powerful language, but a much more accessible language, and it comfortably covers the 95%+ use cases while still being usable by 95%+ of its users?
>
My main objection with create_resources function is that it is not a
>> We believe that the ability to have indirection for the type is a better solution than the alternatives, even if looks more complex, it makes the logic visible and reduces the dependency on Ruby.
>
> I am not convinced. Based on what I've seen so far, I prefer transformation to a hash, and using a function. This is a lot of syntactical power, and it is being done to replace what seems like relatively simple hash operations and a simple function.
>
> I know this wouldn't cover every case, but wouldn't it cover the majority of cases, and thus be a better trade-off? That is, we have a somewhat less powerful language, but a much more accessible language, and it comfortably covers the 95%+ use cases while still being usable by 95%+ of its users?
>
natural progression from the language. When developing puppet code, you
start out with simple resources and use the syntax for creating them. As
you are building up your modules and complexity increases you reach a
point where you have to redo what you have already done because now you
have to instead construct a hash and call a function.
When you have reached this point several times, you are more likely to
always use create_resources.
When instead directly supported in the language, you can add the more
advanced things if and when they are needed.
Having the power to do so, does not take anything away.
Andy Parker
Aug 4, 2014, 11:56:50 AM8/4/14
to puppe...@googlegroups.com
Many of the cases where things have changed are documented at http://docs.puppetlabs.com/puppet/3.6/reference/experiments_future.html
As Henrik said in an earlier reply the "we have to" changes mostly came from working to make things behave in the way people have expressed that they expect it to behave. Sometimes these were changes to interpretation of some element of the language, such as undef, which has fairly localized effects. Others have had impacts on the entire grammar (such as allowing literals in certain places).
[...]
Another case that we had in mind when putting this together was how to handle defaults. Since we are not changing the resource default expression, there is a problem where you want defaults local to a scope (class or define), but don't want to screw up other things in the dynamic scope by using the resource defaults expression. The *=> allows manifest authors to handle this case without large amounts of duplication:
$file_defaults = { owner => puppet, group => puppet }
...
file { default: *=> $file_defaults; ... }
...
file { default: *=> $file_defaults; ... }
I know this wouldn't cover every case, but wouldn't it cover the majority of cases, and thus be a better trade-off? That is, we have a somewhat less powerful language, but a much more accessible language, and it comfortably covers the 95%+ use cases while still being usable by 95%+ of its users?
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/7BA49B49-5B61-4C2A-8EA4-C431B92811AF%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.
Reid Vandewiele
Aug 4, 2014, 12:09:24 PM8/4/14
to puppe...@googlegroups.com
On Sunday, August 3, 2014 4:32:39 PM UTC-7, henrik lindberg wrote:
My main objection with create_resources function is that it is not a
natural progression from the language. When developing puppet code, you
start out with simple resources and use the syntax for creating them. As
you are building up your modules and complexity increases you reach a
point where you have to redo what you have already done because now you
have to instead construct a hash and call a function.
When you have reached this point several times, you are more likely to
always use create_resources.
When instead directly supported in the language, you can add the more
advanced things if and when they are needed.
Having the power to do so, does not take anything away.
- henrik
I am concerned that adding additional operators to the language does in fact take something away. As such, I do think that proposals to do so need to require a very strong argument in order to proceed.
When I go out and introduce new teams of sysadmins to Puppet, the fact in vacuo that we have our own domain-specific language is anything but a strength. One of the top initial objections to Puppet today is that large teams will not be able, or will not want, to learn a new, complex language. What sets us apart from competitors like Chef, from scripting alternatives, is the simplicity and apparent intuitive design of our DSL. The fact that it is more akin to a configuration file than to a procedural program. That a sysadmin can look at a Puppet manifest and usually figure out the basics of what it does and even modify it without needing a training course or a pocket reference. Every operator that is added to the language has the potential to take away from that, and so absolutely needs to be approached as a design decision focused on the end-user experience.
We need the ability to better transform structured data into resources. From an end-user experience perspective (setting aside implementation considerations) I much prefer the idea of using a hash directly over introducing a new operator. The difference would be the two examples shown below.
Proposed operator (proposed):
$x.each |$title, $attributes| { file { $title: * => $attributes } }
Formal hash treatment (my preferred):
$x.each |$title, $attributes| { file { $title: $attributes } }
If we introduce a new operator it should be a user-experience design decision, not an implementation decision. If the preferred design cannot be implemented due to technical constraints that's something we have to deal with. But I hope that we're arriving at decisions to introduce new operators as part of an intentional end-user experience focused design.
The fact that we have a DSL is not a strength. The fact that it is simple and intuitive to practitioners in this space, is.
Ashley Penney
Aug 4, 2014, 12:22:15 PM8/4/14
to puppe...@googlegroups.com
On Mon, Aug 4, 2014 at 12:09 PM, Reid Vandewiele <re...@puppetlabs.com> wrote:
I am concerned that adding additional operators to the language does in fact take something away. As such, I do think that proposals to do so need to require a very strong argument in order to proceed.When I go out and introduce new teams of sysadmins to Puppet, the fact in vacuo that we have our own domain-specific language is anything but a strength. One of the top initial objections to Puppet today is that large teams will not be able, or will not want, to learn a new, complex language. What sets us apart from competitors like Chef, from scripting alternatives, is the simplicity and apparent intuitive design of our DSL. The fact that it is more akin to a configuration file than to a procedural program. That a sysadmin can look at a Puppet manifest and usually figure out the basics of what it does and even modify it without needing a training course or a pocket reference. Every operator that is added to the language has the potential to take away from that, and so absolutely needs to be approached as a design decision focused on the end-user experience.We need the ability to better transform structured data into resources. From an end-user experience perspective (setting aside implementation considerations) I much prefer the idea of using a hash directly over introducing a new operator. The difference would be the two examples shown below.Proposed operator (proposed):$x.each |$title, $attributes| { file { $title: * => $attributes } }Formal hash treatment (my preferred):$x.each |$title, $attributes| { file { $title: $attributes } }If we introduce a new operator it should be a user-experience design decision, not an implementation decision. If the preferred design cannot be implemented due to technical constraints that's something we have to deal with. But I hope that we're arriving at decisions to introduce new operators as part of an intentional end-user experience focused design.The fact that we have a DSL is not a strength. The fact that it is simple and intuitive to practitioners in this space, is.
I've been unable to articulate my concerns with the current language discussions but I think Reid has done a good enough job that I feel comfortable contributing. It seems like all the talk is now about the "Puppet language" instead of the "DSL" and this is, in my opinion, a mistake.
We seem to be trying to build a general purpose language that is powerful enough to write other things in, such as functions, types and providers, etc. We already have languages for those and I don't think that we can build a better language internally than these external languages.
As a member of the module team I honestly haven't understood a large part of the proposed enhancements over the last year, or really understood the need for them. There seems to be a bit of a disconnect between people who write modules and people who like languages in a lot of the conversations.
Some enhancements, like improvements to defaults, make sense. Things like putting resources into $a and then being able to do things with them just make the DSL much more difficult to teach, reason able, and expose to systems administrators. I too prefer the hash to a * operator, it's follows the patterns of the DSL that we've been using for years and (to my non-developer brain) makes more sense.
I've tried to raise these concerns in the past but I've done a bad job of talking to platform people about it. I think that the separation between people building the language and people writing the modules has the potential to do us a lot of harm. At the very least all these language proposals and suggestions and syntax changes should come with "real production code" examples that show how you solve it today and how this proposed change leads to a more elegant situation for actual puppet users because right now it seems to be happening in isolation from the people using it to build things.
--
Ashley Penney
Module EngineerJoin us at PuppetConf 2014, September 23-24 in San Francisco - http://puppetconf.com
Trevor Vaughan
Aug 4, 2014, 2:35:09 PM8/4/14
to puppe...@googlegroups.com
So, I'm chiming in to say that I completely agree with Reid and Ashley.
As an end user, I want to hand off code that is clear and relatively easy to read. I definitely do not want magic symbols (or I would have stuck with PERL).
I'm OK with all of the concepts proposed but I would like more verbosity and clarity as opposed to more 'elegance' and mystery.
Thanks,
Trevor
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAC9eg%2BkFefgHzYJmYU5k9%3DF-EhMj3h3y9O9qNGDQigKpzx0ewA%40mail.gmail.com.
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com
-- This account not approved for unencrypted proprietary information --
Henrik Lindberg
Aug 4, 2014, 6:18:58 PM8/4/14
to puppe...@googlegroups.com
On 2014-04-08 20:35, Trevor Vaughan wrote:
> So, I'm chiming in to say that I completely agree with Reid and Ashley.
>
> As an end user, I want to hand off code that is clear and relatively
> easy to read. I definitely do not want magic symbols (or I would have
> stuck with PERL).
>
> I'm OK with all of the concepts proposed but I would like more verbosity
> and clarity as opposed to more 'elegance' and mystery.
>
So, to summarize: The use of * => as an operator is not liked but the
> So, I'm chiming in to say that I completely agree with Reid and Ashley.
>
> As an end user, I want to hand off code that is clear and relatively
> easy to read. I definitely do not want magic symbols (or I would have
> stuck with PERL).
>
> I'm OK with all of the concepts proposed but I would like more verbosity
> and clarity as opposed to more 'elegance' and mystery.
>
concept of being able to set attributes from a hash is. Unfortunately,
it is not possible to directly allow an expression at the position in
question, there must be a syntactical marker.
As pointed out earlier, the * => was thought to read as "any_attribute
=> from_these_values", but I totally grok if people have an allergic
reaction.
We can do this though:
file { default: ($hash) }
This works because it is impossible to have an attribute name in
parentheses.
In use:
file (
default : ($my_file_defaults + { mode => '0666' });
'/tmp/foo': ;
'/tmp/bar': ;
}
Is that better? No new operator, but you have to use parentheses around
the expression.
We can naturally also revert the functionality, but it seems it is liked
conceptually.
Trevor Vaughan
Aug 4, 2014, 7:08:44 PM8/4/14
to puppe...@googlegroups.com
This is certainly better than the new operator but the parenthesis are a bit strange.
Isn't the title 'default' special enough to not have the parenthesis?
If not, then I can certainly live with the parenthesis syntax better than a new operator (though support in the syntax checkers will be most useful).
Thanks,
Trevor
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lrp0s2%24c09%241%40ger.gmane.org.
Henrik Lindberg
Aug 4, 2014, 7:17:04 PM8/4/14
to puppe...@googlegroups.com
On 2014-05-08 1:08, Trevor Vaughan wrote:
> This is certainly better than the new operator but the parenthesis are a
> bit strange.
>
> Isn't the title 'default' special enough to not have the parenthesis?
>
The functionality is not limited to the default title, and a literal
> This is certainly better than the new operator but the parenthesis are a
> bit strange.
>
> Isn't the title 'default' special enough to not have the parenthesis?
>
default is not a special expression so the grammar has no special
knowledge. It is basically:
Body : expression ':' attribute_operations ... ;
i.e. you can use it for a titled resource as well:
file { '/tmp/foo': ($hash) }
> If not, then I can certainly live with the parenthesis syntax better
> than a new operator (though support in the syntax checkers will be most
> useful).
>
validation of this - it cannot validate that the expression will
evaluate to a valid hash though, that is not known until you actually
run the code). But maybe you meant something else?
Regards
- henrik
> Thanks,
>
> Trevor
>
>
> On Mon, Aug 4, 2014 at 6:18 PM, Henrik Lindberg
> <http://puppet-on-the-edge.blogspot.se/>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to puppet-dev+unsubscribe@__googlegroups.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> <mailto:puppet-dev%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/__msgid/puppet-dev/lrp0s2%24c09%__241%40ger.gmane.org
> <https://groups.google.com/d/msgid/puppet-dev/lrp0s2%24c09%241%40ger.gmane.org>.
>
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
> tvau...@onyxpoint.com <mailto:tvau...@onyxpoint.com>
>
> -- This account not approved for unencrypted proprietary information --
>
> -- This account not approved for unencrypted proprietary information --
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> <mailto:puppet-dev+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUGYSACC5OS0vkyZVT6Yz6j2kgaGGphZmis90mWKSYKmA%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUGYSACC5OS0vkyZVT6Yz6j2kgaGGphZmis90mWKSYKmA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Trevor Vaughan
Aug 4, 2014, 8:37:15 PM8/4/14
to puppe...@googlegroups.com
Sorry for the vagueness. I was referring to Geppetto, the Vim plugin, etc...
Random parenthesis are something that I think will be often overlooked without some help.
Thanks,
Trevor
On Mon, Aug 4, 2014 at 7:16 PM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
On 2014-05-08 1:08, Trevor Vaughan wrote:The functionality is not limited to the default title, and a literal default is not a special expression so the grammar has no special knowledge. It is basically:
This is certainly better than the new operator but the parenthesis are a
bit strange.
Isn't the title 'default' special enough to not have the parenthesis?
Body : expression ':' attribute_operations ... ;
i.e. you can use it for a titled resource as well:
file { '/tmp/foo': ($hash) }You mean in puppet parser validate? Or where? (You should get static validation of this - it cannot validate that the expression will evaluate to a valid hash though, that is not known until you actually run the code). But maybe you meant something else?
If not, then I can certainly live with the parenthesis syntax better
than a new operator (though support in the syntax checkers will be most
useful).
Regards
- henrik
Thanks,
Trevor
On Mon, Aug 4, 2014 at 6:18 PM, Henrik Lindberg
an email to puppet-dev+unsubscribe@googlegroups.com<mailto:puppet-dev+unsub...@googlegroups.com>.
https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUGYSACC5OS0vkyZVT6Yz6j2kgaGGphZmis90mWKSYKmA%40mail.gmail.com
To view this discussion on the web visit
<https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUGYSACC5OS0vkyZVT6Yz6j2kgaGGphZmis90mWKSYKmA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lrp48o%24i8b%241%40ger.gmane.org.
Erik Dalén
Aug 5, 2014, 6:19:12 AM8/5/14
to Puppet Developers
As an example where this kind of expressiveness is needed: https://github.com/spotify/puppet-puppetexplorer/blob/master/manifests/init.pp#L89-L97
Here I used it to allow the user to simply pass a hash with any additional apache::vhost parameters as a parameter instead of exposing all of them (and the usage of the hash() function is because puppet 3.x doesn't allow variables as keys in hashes...).
But either of those proposed solutions could be used there I guess (*=> or default: pointing to a hash)
The first proposal was:
title: (expr)
Then, just use the splat operator:
title: *expr
But then it became weird, because it is a special kind of splat that
applies to the entire expression - that felt more confusing.
But haven't you still got a special kind of splat? I think all you've done is alter the spelling to distinguish it from the other splat -- which is better than not -- but it's still kinda confusing.Agreed.
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/D4F608CD-67BF-4E4C-BA68-E54BF1A10678%40puppetlabs.com.
Erik Dalén
Reid Vandewiele
Aug 5, 2014, 10:26:14 AM8/5/14
to puppe...@googlegroups.com
On Mon, Aug 4, 2014 at 3:18 PM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
So, to summarize: The use of * => as an operator is not liked but the concept of being able to set attributes from a hash is. Unfortunately, it is not possible to directly allow an expression at the position in question, there must be a syntactical marker.
As pointed out earlier, the * => was thought to read as "any_attribute => from_these_values", but I totally grok if people have an allergic reaction.
We can do this though:
file { default: ($hash) }
This works because it is impossible to have an attribute name in parentheses.
In use:
file (
default : ($my_file_defaults + { mode => '0666' });
'/tmp/foo': ;
'/tmp/bar': ;
}
Is that better? No new operator, but you have to use parentheses around the expression.
We can naturally also revert the functionality, but it seems it is liked conceptually.
- henrik
I think the parenthesis are far preferable over *=>. That isn't to say I like them - I don't particularly. But the functionality is desirable, and if it's a matter of a technical limitation then parenthesis are a Good Enough (TM) compromise from the more ideal direct use of a hash.
~Reid
Erik Dalén
Aug 5, 2014, 11:18:52 AM8/5/14
to Puppet Developers
I really prefer the use of * => over the parenthesis. Especially if you need to merge things into the hash. For example look at this example:
# using parenthesis hash styleclass foo ($servername = $::fqdn,$port = 80,$ssl = false,$extra_opts={},) {apache::vhost { $servername: ($extra_opts + {port => $port,ssl => $ssl,})}}# using * =>class foo ($servername = $::fqdn,$port = 80,$ssl = false,$extra_opts={},) {apache::vhost { $servername:port => $port,ssl => $ssl,* => $extra_opts,}
}
IMO the *=> is way more readable (as gist here if formatting is screwed up for you: https://gist.github.com/dalen/57b37b80a9ba1879b78c). This is quite similar to what I linked earlier that I am doing in https://github.com/spotify/puppet-puppetexplorer/blob/master/manifests/init.pp#L89-L97
So it is not just a contrived example.
Erik Dalén
Andy Parker
Aug 5, 2014, 12:24:40 PM8/5/14
to puppe...@googlegroups.com
The behavior that we worked out doesn't allow having the splat along with the other parameters, the reason being that it isn't clear what is meant by that. You had in your head that port and ssl are overridden by extra_opts (possibly because they come first?), but another, completely reasonable interpretation is that it is the other way around (port and ssl override extra_opts because they are explicitly given. In order to remove any of that confusion the current specification and implementation doesn't allow mixing. That can, I think, be changed.
In the current implementation this would need to be:
apache::vhost { $servername:
* => $extra_opts + { port => $port, ssl => $ssl }
}
IMO the *=> is way more readable (as gist here if formatting is screwed up for you: https://gist.github.com/dalen/57b37b80a9ba1879b78c). This is quite similar to what I linked earlier that I am doing in https://github.com/spotify/puppet-puppetexplorer/blob/master/manifests/init.pp#L89-L97So it is not just a contrived example.
My argument against using parenthesis is that parenthesis, are often read as "seldom necessary grouping". I believe that most programmers read them as usually only needed for fixing precedence problems, which is really what is happening here but it doesn't look like it. Based on that I can imagine that a common, and frustrating mistake would be:
apache::vhost { $servername: $opts }
And then confusion and anger and bug reports.
--
Erik Dalén
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAAAzDLdQv5iC40X0Qpx9pFaEhKf6FBPwf%3DKoLjtO7oDnDVt5bw%40mail.gmail.com.
David Schmitt
Aug 5, 2014, 2:52:26 PM8/5/14
to puppe...@googlegroups.com
On 2014-08-05 12:19, Erik Dalén wrote:
>
>
>
> On 3 August 2014 22:18, Luke Kanies <lu...@puppetlabs.com
>
>
>
> On 3 August 2014 22:18, Luke Kanies <lu...@puppetlabs.com
> <john.bo...@stjude.org <mailto:john.bo...@stjude.org>> wrote:
>
>>
>>
>> On Friday, July 25, 2014 5:15:30 PM UTC-5, henrik lindberg wrote:
>>
>>
>> We reasoned that we already have create_resources in frequent
>> use to
>> solve real issues, so it is needed. I don't think
>> create_resources is
>> used simply because you can. Instead, it is because you do not
>> statically know the type, or you want to decide which
>> attributes to set
>> based on calculations you made. Both of those require
>> create_resources
>> function. Now it moves into the language.
>>
>>
>>
>> For what it's worth, every use of create_resources() that I have
>> ever come across or thought about has involved a resource type
>> that is statically known. The usual reason for create_resources()
>> is that the *identities* and properties of the resources need to
>
>>
>>
>> On Friday, July 25, 2014 5:15:30 PM UTC-5, henrik lindberg wrote:
>>
>>
>> We reasoned that we already have create_resources in frequent
>> use to
>> solve real issues, so it is needed. I don't think
>> create_resources is
>> used simply because you can. Instead, it is because you do not
>> statically know the type, or you want to decide which
>> attributes to set
>> based on calculations you made. Both of those require
>> create_resources
>> function. Now it moves into the language.
>>
>>
>>
>> For what it's worth, every use of create_resources() that I have
>> ever come across or thought about has involved a resource type
>> that is statically known. The usual reason for create_resources()
>> be dynamically determined. That is, if I need or want to resort
>> to create_resources() then it's /usually/ because I don't know
>> statically which specific resources (of a known type) need to be
>> managed. Sometimes I /also/ don't know which properties need to
that $vhost_options is not allowed to override the base_vhost_options
and give a reason for that. I needed to browse up to the parameter doc
and think a bit about what that should mean.
I do not think the whole sequence would be any better with some kind of
special operator, except perhaps for the hash() thingy, which I
conveniently ignore in the analysis, but assume it's doing some merging.
Also, create_resources is google-able. To find the splat operator, one
would either have to know it or think about the language reference and
browse through the visual index or the operator chapter.
>> The first proposal was:
>>
>> title: (expr)
>>
>> Then, just use the splat operator:
>>
>> title: *expr
>>
>> But then it became weird, because it is a special kind of
>> splat that
>> applies to the entire expression - that felt more confusing.
>>
>>
>>
>> But haven't you still got a special kind of splat? I think all
>> you've done is alter the spelling to distinguish it from the other
>> splat -- which is better than not -- but it's still kinda confusing.
>
> Agreed.
>
>
> --
> http://puppetlabs.com/ | http://about.me/lak | @puppetmasterd
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to puppet-dev+...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/D4F608CD-67BF-4E4C-BA68-E54BF1A10678%40puppetlabs.com
> <https://groups.google.com/d/msgid/puppet-dev/D4F608CD-67BF-4E4C-BA68-E54BF1A10678%40puppetlabs.com?utm_medium=email&utm_source=footer>.
> https://groups.google.com/d/msgid/puppet-dev/D4F608CD-67BF-4E4C-BA68-E54BF1A10678%40puppetlabs.com
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CAAAzDLcUss5MsGEvZsSUJUtxFdQ6vK%3Dr1e1wC33fB24YMXXAow%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CAAAzDLcUss5MsGEvZsSUJUtxFdQ6vK%3Dr1e1wC33fB24YMXXAow%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Henrik Lindberg
Aug 5, 2014, 7:12:11 PM8/5/14
to puppe...@googlegroups.com
On 2014-05-08 18:24, Andy Parker wrote:
> On Tue, Aug 5, 2014 at 8:18 AM, Erik Dalén <erik.gus...@gmail.com
> <mailto:erik.gus...@gmail.com>> wrote:
>
> On 5 August 2014 16:25, Reid Vandewiele <re...@puppetlabs.com
> On Tue, Aug 5, 2014 at 8:18 AM, Erik Dalén <erik.gus...@gmail.com
> <mailto:erik.gus...@gmail.com>> wrote:
>
> On 5 August 2014 16:25, Reid Vandewiele <re...@puppetlabs.com
> <mailto:re...@puppetlabs.com>> wrote:
>
> On Mon, Aug 4, 2014 at 3:18 PM, Henrik Lindberg
> <henrik....@cloudsmith.com
>
> On Mon, Aug 4, 2014 at 3:18 PM, Henrik Lindberg
> <henrik....@cloudsmith.com
One more proposal :-)
We could leave out the name part all together (i.e. drop the '*').
dalens' example would then look like this:
apache::vhost { $servername:
port => $port,
ssl => $ssl,
And if it is used for local defaults (or the only thing for a titled
resource):
file { default: => $hash }
file { '/tmp/foo': => $hash }
This works best if it is restricted to being the only attribute
operation for a title, but looks a bit odd when presented in a list
where there are also named (i.e. name => expression) operations.
At least it is not a new operator.
Is this better than * => or requiring parentheses ?
Reid Vandewiele
Aug 5, 2014, 9:41:40 PM8/5/14
to puppe...@googlegroups.com
I'm still not happy with either "* =>" or " =>". Both unnecessarily (imho) complicate the structure of the most basic building block in the Puppet language.
I like that piece of code as it is. Perhaps I would add a comment noting that $vhost_options is not allowed to override the base_vhost_options and give a reason for that. I needed to browse up to the parameter doc and think a bit about what that should mean.
I do not think the whole sequence would be any better with some kind of special operator, except perhaps for the hash() thingy, which I conveniently ignore in the analysis, but assume it's doing some merging.
Also, create_resources is google-able. To find the splat operator, one would either have to know it or think about the language reference and browse through the visual index or the operator chapter.
Maybe solving for this use case would be better handled by implementing something that looks and feels like a metaparameter rather than trying to come up with new syntax. That approach would have the benefit of not complicating the language, and meet all of the functional requirements discussed so far. It would also be google-able. There would need to be some design around the choice of a name for the "metaparameter", but it's easy enough to demonstrate the concept with a stand-in like "attribute_defaults" or "attribute_hash".
Example 1 (assuming behavior whereinmerging is OK, and that explicit parameter specification takes precedence):
apache::vhost { $servername:
port => $port,
ssl => $ssl,
attribute_defaults => $extra_opts,
}
Example 2 (assuming that merging is not OK, and that conflicts will be treated as duplicate parameter specification):
apache::vhost { $servername:
port => $port,
ssl => $ssl,
attribute_hash => $extra_opts,
}
My initial thought would be to choose and settle on one behavior and review an appropriate name, though it wouldn't be objectionable to support both.
Does an operator/syntax gain us anything that this kind of metaparameter-like approach does not?
Is taking a metaparameter-like approach still a language feature, or does that become an actual metaparameter?
Visual review, for convenience:
file { $title: * => $attributes; }
file { $title: => $attributes; }
file { $title: ($attributes); }
file { $title: attribute_defaults => $attributes; }
file { $title: attribute_hash => $attributes; }
~Reid
Henrik Lindberg
Aug 6, 2014, 9:47:05 AM8/6/14
to puppe...@googlegroups.com
On 2014-06-08 3:41, Reid Vandewiele wrote:
> On Tue, Aug 5, 2014 at 4:11 PM, Henrik Lindberg
> <henrik....@cloudsmith.com <mailto:henrik....@cloudsmith.com>>
> On Tue, Aug 5, 2014 at 4:11 PM, Henrik Lindberg
it requires a special reserved name. Currently attribute names can be
any valid name including keywords (except true and false, plus the names
that are already reserved for meta parameters).
An operator does not add any other value.
A small difference is that the meta attribute name may be used in
manifests that are evaluated by an older puppet. In those cases there
would be an error that "attribute_hash" is not a valid attribute name,
as opposed to a syntax error (if an operator is used).
> Is taking a metaparameter-like approach still a language feature, or
> does that become an actual metaparameter?
>
ripples through the entire system.
> Visual review, for convenience:
>
> file { $title: * => $attributes; }
> file { $title: => $attributes; }
> file { $title: ($attributes); }
> file { $title: attribute_defaults => $attributes; }
> file { $title: attribute_hash => $attributes; }
>
Spencer Krum
Aug 6, 2014, 10:56:24 AM8/6/14
to puppe...@googlegroups.com
I think a metaparameter is a great solution. It has the advantage of feeling the most puppety, and can be successfully googled for.
I don't think I agree that attribute_hash is more descriptive than attribute_defaults, since, aren't we setting defaults?
Thanks,
Spencer
I don't think I agree that attribute_hash is more descriptive than attribute_defaults, since, aren't we setting defaults?
Thanks,
Spencer
On Wed, Aug 6, 2014 at 6:46 AM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
On 2014-06-08 3:41, Reid Vandewiele wrote:
On Tue, Aug 5, 2014 at 4:11 PM, Henrik Lindberg
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lrtbji%24fqe%241%40ger.gmane.org.
Ben Ford
Aug 6, 2014, 11:41:38 AM8/6/14
to puppe...@googlegroups.com
I really like the metaparameter approach. It's almost self documenting.
Ashley Penney
Aug 6, 2014, 11:56:12 AM8/6/14
to puppe...@googlegroups.com
I hate to "me too" but I'm actually a pretty big fan of the metaparam too, it's very readable and clear.
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/1d38a4dd-596e-4338-b0d5-f6117f3a6052%40googlegroups.com.
Wil Cooley
Aug 6, 2014, 1:29:41 PM8/6/14
to puppet-dev group
On Aug 5, 2014 6:41 PM, "Reid Vandewiele" <re...@puppetlabs.com> wrote:
> Example 1 (assuming behavior whereinmerging is OK, and that explicit parameter specification takes precedence):
>
> apache::vhost { $servername:
> port => $port,
> ssl => $ssl,
> attribute_defaults => $extra_opts,
> }
> Example 1 (assuming behavior whereinmerging is OK, and that explicit parameter specification takes precedence):
>
> apache::vhost { $servername:
> port => $port,
> ssl => $ssl,
> attribute_defaults => $extra_opts,
> }
Assuming that the semantics of 'undef' haven't changed to preclude this (I know there are changes coming but don't quite remember what they are), an interesting couple of cases are:
And:
My expectation is that in both cases 'undef' means "pretend I've said nothing about thus attribute here" and that *true* wins in either case.
But what if what is intended is to explicitly *ignore* the setting in $extra_opts, so that the type's default is used, as if the attribute had been mentioned by neither the $extra_opts hash or the apache::vhost declaration? I guess using a hash operation to copy and filter $extra_opts would do it.
$extra_opts = { $ssl => true, ... }
apache::vhost { ...
ssl => undef,
attributes => $extra_opts,
}
And:
$extra_opts = { $ssl => undef, ... }
apache::vhost { ...
ssl => true,
attributes => $extra_opts,
}
My expectation is that in both cases 'undef' means "pretend I've said nothing about thus attribute here" and that *true* wins in either case.
By the way (since the paint's not dry) I think just *attributes* as the afforementioned metaparam would be best -- it clearly says what it is, is short, and is meta-sounding enough that I would not even consider using it "in the field" -- when developing a class, defined type, type/provider, etc.
Another alternative that occurs to me that breaks the "key => val" layout and so is cognitively distinct is to use something like the facilities that many modern languages have for passing hashes or arrays as arguments to methods/functions:
apache::vhost { $title:
...**$extra_opts}
or:
apache::vhost { $title:...&$extra_opts}
or:
apache::vhost { $tilte:...*$extra_opts}
Henrik Lindberg
Aug 6, 2014, 4:15:06 PM8/6/14
to puppe...@googlegroups.com
On 2014-06-08 16:56, Spencer Krum wrote:
> I think a metaparameter is a great solution. It has the advantage of
> feeling the most puppety, and can be successfully googled for.
>
> I don't think I agree that attribute_hash is more descriptive than
> attribute_defaults, since, aren't we setting defaults?
>
No, this is not for defaults, it is for assigning values to attributes.
> I think a metaparameter is a great solution. It has the advantage of
> feeling the most puppety, and can be successfully googled for.
>
> I don't think I agree that attribute_hash is more descriptive than
> attribute_defaults, since, aren't we setting defaults?
>
Local defaults are set by setting attributes of an entry labeled with a
title that is a literal default.
- henrik
> Thanks,
> Spencer
>
>
> On Wed, Aug 6, 2014 at 6:46 AM, Henrik Lindberg
> wrote:
>
> On 2014-06-08 3:41, Reid Vandewiele wrote:
>
> On Tue, Aug 5, 2014 at 4:11 PM, Henrik Lindberg
> <henrik.lindberg@cloudsmith.__com
>
> On 2014-06-08 3:41, Reid Vandewiele wrote:
>
> On Tue, Aug 5, 2014 at 4:11 PM, Henrik Lindberg
> <mailto:henrik....@cloudsmith.com>
> <mailto:henrik.lindberg@__cloudsmith.com
> <mailto:henrik....@cloudsmith.com>>>
> <http://puppet-on-the-edge.blogspot.se/>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to puppet-dev+unsubscribe@__googlegroups.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> <mailto:puppet-dev%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/__msgid/puppet-dev/lrtbji%24fqe%__241%40ger.gmane.org
> <https://groups.google.com/d/msgid/puppet-dev/lrtbji%24fqe%241%40ger.gmane.org>.
>
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
>
>
>
> --
> Spencer Krum
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
>
>
>
> --
> (619)-980-7820
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> <mailto:puppet-dev+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CADt6FWPh4uaR%3DWHJq3JO7eFe9xQ8fugCJWiuSw-v%3DoEXcD0kyg%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CADt6FWPh4uaR%3DWHJq3JO7eFe9xQ8fugCJWiuSw-v%3DoEXcD0kyg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Henrik Lindberg
Aug 6, 2014, 4:30:47 PM8/6/14
to puppe...@googlegroups.com
On 2014-06-08 19:29, Wil Cooley wrote:
> On Aug 5, 2014 6:41 PM, "Reid Vandewiele" <re...@puppetlabs.com
> On Aug 5, 2014 6:41 PM, "Reid Vandewiele" <re...@puppetlabs.com
> <mailto:re...@puppetlabs.com>> wrote:
>
> > Example 1 (assuming behavior whereinmerging is OK, and that explicit
> parameter specification takes precedence):
> >
> > apache::vhost { $servername:
> > port => $port,
> > ssl => $ssl,
> > attribute_defaults => $extra_opts,
> > }
>
> Assuming that the semantics of 'undef' haven't changed to preclude this
> (I know there are changes coming but don't quite remember what they
> are), an interesting couple of cases are:
>
> $extra_opts = { $ssl => true, ... }
I assume you mean {ssl => true, ...}, and not the value of a variable $ssl ?
>
> > Example 1 (assuming behavior whereinmerging is OK, and that explicit
> parameter specification takes precedence):
> >
> > apache::vhost { $servername:
> > port => $port,
> > ssl => $ssl,
> > attribute_defaults => $extra_opts,
> > }
>
> Assuming that the semantics of 'undef' haven't changed to preclude this
> (I know there are changes coming but don't quite remember what they
> are), an interesting couple of cases are:
>
> $extra_opts = { $ssl => true, ... }
> apache::vhost { ...
> ssl => undef,
> attributes => $extra_opts,
> }
>
>
> And:
>
> $extra_opts = { $ssl => undef, ... }
> apache::vhost { ...
> ssl => true,
> attributes => $extra_opts,
> }
>
(most likely a mistake).
>
> My expectation is that in both cases 'undef' means "pretend I've said
> nothing about thus attribute here" and that *true* wins in either case.
>
attribute; it then means "No value specified for this parameter".
That is undef behaves as a "value" until the resource creation takes place.
> But what if what is intended is to explicitly *ignore* the setting in
> $extra_opts, so that the type's default is used, as if the attribute had
> been mentioned by neither the $extra_opts hash or the apache::vhost
> declaration? I guess using a hash operation to copy and filter
> $extra_opts would do it.
>
create a new hash where one or several keys are removed. Like this:
$a = { a=> 10, b => 20, c => 30}
$b = $a - [a, b] # $b is set to {c => 30 }
> By the way (since the paint's not dry) I think just *attributes* as the
> afforementioned metaparam would be best -- it clearly says what it is,
> is short, and is meta-sounding enough that I would not even consider
> using it "in the field" -- when developing a class, defined type,
> type/provider, etc.
>
> Another alternative that occurs to me that breaks the "key => val"
> layout and so is cognitively distinct is to use something like the
> facilities that many modern languages have for passing hashes or arrays
> as arguments to methods/functions:
>
> apache::vhost { $title:
> ...
> **$extra_opts
> }
>
>
> or:
>
> apache::vhost { $title:
> ...
> &$extra_opts
> }
>
>
> or:
>
> apache::vhost { $tilte:
> ...
> *$extra_opts
> }
>
the value. This also looks similar to the unary splat '*' that can be
used to unfold an array or hash.
That suggestion was not well liked... "no more operators please...".
It would be possible to specify '*' as signaling "here is an attributes
hash". The only quirk is that it would be a special syntactic marker
that is followed by any expression (which includes splat) - no harm,
just slightly odd that you are allowed to write **$extra_opts, where
this is not allowed elsewhere (splat is non associative, just like unary
minus).
- henrik
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CAMmm3r557mJAxXT0m3vXv6T9OMfQCwK9npkdnT7Ro7ttkGtgUA%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CAMmm3r557mJAxXT0m3vXv6T9OMfQCwK9npkdnT7Ro7ttkGtgUA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Wil Cooley
Aug 6, 2014, 7:19:02 PM8/6/14
to puppet-dev group
On Wed, Aug 6, 2014 at 1:30 PM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
$extra_opts = { $ssl => true, ... }
I assume you mean {ssl => true, ...}, and not the value of a variable $ssl ?
Yes, sorry
Both of these results in an error, you are not allowed to set ssl twice (most likely a mistake).
apache::vhost { ...
ssl => undef,
attributes => $extra_opts,
}
And:
$extra_opts = { $ssl => undef, ... }
apache::vhost { ...
ssl => true,
attributes => $extra_opts,
}
I think I trimmed too much context; Reid had proffered two behaviors and I was responding to the first:
Example 1 (assuming behavior whereinmerging is OK, and that explicit parameter specification takes precedence):
...
Example 2 (assuming that merging is not OK, and that conflicts will be treated as duplicate parameter specification):
Example 1 can be used to replace/supplement resource defaults; 2 cannot.
undef only has a special meaning when it is the final value of the given attribute; it then means "No value specified for this parameter".My expectation is that in both cases 'undef' means "pretend I've said
nothing about thus attribute here" and that *true* wins in either case.
That is undef behaves as a "value" until the resource creation takes place.
I think that amounts to the same thing, but from different sides of the compiler, so to speak.
Such as ignoring a resource default:
Exec { timeout => 30 }
# Bunch of execs that sometimes take too damn long
exec { '/usr/bin/sleep 60': }
...
# Then another exec that should have the default timeout, but I don't want# to hardcode (or even remember) what that is (i.e. pretend the above# resource default does not exist):
Or passing options from a defined type to a type declaration:exec { '/usr/bin/ping localhost': timeout => undef }
define filelike($seltype = undef) {
file { '/tmp/foo':
seltype => $seltype,}}
apache::vhost { $title: **$extra_opts }
or:
apache::vhost { $title: &$extra_opts }
or:
apache::vhost { $title: *$extra_opts }
So, that was the initial proposal '*' for any name followed by => and the value. This also looks similar to the unary splat '*' that can be used to unfold an array or hash.
That suggestion was not well liked... "no more operators please...".
Part of what is bothersome to me about the "* =>" proposal is that it looks like
an assignment to a wildcard, which makes my head spin.
It would be possible to specify '*' as signaling "here is an attributes hash". The only quirk is that it would be a special syntactic marker that is followed by any expression (which includes splat) - no harm, just slightly odd that you are allowed to write **$extra_opts, where this is not allowed elsewhere (splat is non associative, just like unary minus).
Rather than a special construct that only applied for attributes, it could be a
general unpacking operator, like it is in Ruby and Python. I guess that's a bigger can of worms.
I echo Reid's sentiments about language complexity and prefer the metavar.
Wil
Henrik Lindberg
Aug 6, 2014, 8:15:32 PM8/6/14
to puppe...@googlegroups.com
On 2014-07-08 1:18, Wil Cooley wrote:
> On Wed, Aug 6, 2014 at 1:30 PM, Henrik Lindberg
> <henrik....@cloudsmith.com <mailto:henrik....@cloudsmith.com>>
> On Wed, Aug 6, 2014 at 1:30 PM, Henrik Lindberg
>
>
> apache::vhost { $title: **$extra_opts }
>
>
> or:
>
> apache::vhost { $title: &$extra_opts }
>
> or:
>
> apache::vhost { $title: *$extra_opts }
>
>
> So, that was the initial proposal '*' for any name followed by =>
> and the value. This also looks similar to the unary splat '*' that
> can be used to unfold an array or hash.
>
> That suggestion was not well liked... "no more operators please...".
>
>
> Part of what is bothersome to me about the "* =>" proposal is that it
> looks like
> an assignment to a wildcard, which makes my head spin.
>
> It would be possible to specify '*' as signaling "here is an
> attributes hash". The only quirk is that it would be a special
> syntactic marker that is followed by any expression (which includes
> splat) - no harm, just slightly odd that you are allowed to write
> **$extra_opts, where this is not allowed elsewhere (splat is non
> associative, just like unary minus).
>
>
> Rather than a special construct that only applied for attributes, it
> could be a
> general unpacking operator, like it is in Ruby and Python. I guess
> that's a bigger can of worms.
>
wrote above. The problem in the grammar is that there is the need to
have a syntactic marker, it is impossible to allow any expression at
that point.
> I echo Reid's sentiments about language complexity and prefer the metavar.
>
> Wil
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CAMmm3r5jAUnP0s_rWVqtUqJ4cZjnuCrLLG77XteG0q7rY7W43Q%40mail.gmail.com
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.
> To view this discussion on the web visit
> <https://groups.google.com/d/msgid/puppet-dev/CAMmm3r5jAUnP0s_rWVqtUqJ4cZjnuCrLLG77XteG0q7rY7W43Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Ben Ford
Aug 7, 2014, 2:09:25 PM8/7/14
to puppe...@googlegroups.com
I would also like to propose that we use the metaparameter syntax for defaults as well.
For example, rather than writing this:
For example, rather than writing this:
#less readablefile {default:mode => '0666';'/tmp/foo': ;'/tmp/bar': ;'/tmp/special':mode => '0777';}One would write this:
#more readable$defaults = {ensure => file,mode => '0666',}file { [ '/tmp/foo', '/tmp/bar' ]:attribute_defaults => $defaults,}file { '/tmp/special':mode => '0777',attribute_defaults => $defaults,}The benefits to that approach are that it looks & feels more like the Puppet language, it's less of a conceptual change going forward, and it's also easily searchable. It's also possible have multiple default hash variables in scope and just assign them as needed.
On Wed, Aug 6, 2014 at 5:15 PM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
On 2014-07-08 1:18, Wil Cooley wrote:
On Wed, Aug 6, 2014 at 1:30 PM, Henrik Lindberg
an email to puppet-dev+unsubscribe@googlegroups.com<mailto:puppet-dev+unsub...@googlegroups.com>.
<https://groups.google.com/d/msgid/puppet-dev/CAMmm3r5jAUnP0s_rWVqtUqJ4cZjnuCrLLG77XteG0q7rY7W43Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/CAMmm3r5jAUnP0s_rWVqtUqJ4cZjnuCrLLG77XteG0q7rY7W43Q%40mail.gmail.com
--
Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/lrugeh%24527%241%40ger.gmane.org.
Ben Ford | Training Solutions Engineer
Puppet Labs, Inc.
926 NW 13th Ave, Suite #210
Portland, OR 97209
509.592.7291
ben....@puppetlabs.com
Puppet Labs, Inc.
926 NW 13th Ave, Suite #210
Portland, OR 97209
509.592.7291
ben....@puppetlabs.com
Henrik Lindberg
Aug 7, 2014, 3:55:43 PM8/7/14
to puppe...@googlegroups.com
Your example can also be written like this:
file { [ '/tmp/foo', '/tmp/bar' ]:
attribute => $defaults,
}
file { '/tmp/special':
mode => '0777',
attributes => $defaults - mode,
file { '/tmp/special':
mode => '0777',
}
- henrik
Joshua Partlow
Aug 7, 2014, 4:13:17 PM8/7/14
to puppe...@googlegroups.com
I was muddling through a similar email, but Henrik beat me to the point; the only thing I have to add is that I find attribute_defaults more confusing than an explicit defaults: section because of the added confusion regarding how the attributes mesh. Explicit attributes and attributes => $more would raise an error if they set the same attributes, but attribute_defaults => $defaults would presumably be overwritten.
Josh Partlow
jpar...@puppetlabs.com
Developer, Puppet Labs
Join us at PuppetConf 2014, September 20-24 in San Francisco
Register by September 8th to take advantage of the Final Countdown —save $149!
Luke Kanies
Aug 12, 2014, 1:47:39 PM8/12/14
to puppe...@googlegroups.com
On Aug 5, 2014, at 6:41 PM, Reid Vandewiele <re...@puppetlabs.com> wrote:
Visual review, for convenience:file { $title: * => $attributes; }file { $title: => $attributes; }file { $title: ($attributes); }file { $title: attribute_defaults => $attributes; }file { $title: attribute_hash => $attributes; }
Sorry for joining the party late; I’m clearly not quite on the same email cadence. :/
I think there are some other interesting middle grounds. In particular, we could add explicit merge operations, like other languages have for hashes:
# Make some functions for merging a hash into a resource
file { $title: mode => 644 }.merge($attributes) #destructively merge
file { $title: mode => 644 }.default($attributes) #prefer specified attributes
# Alternative: Make operators for merging
file { $title: mode => 644 } & $attributes # Or something like that
In other words, get rid of the whole concept of explicitly specifying defaults, and also get rid of anything inside the curlies, and just modify the resource in place with a function or operator.
In my head, the bitwise operators seemed like an obvious operator set to use; in practice, it looks, um, questionable at best. At the least, any of the above statements would be expected to affect the resource, and it’s tough to say that’s obvious from the syntax.
I do like the explicitness of it, though.
Henrik Lindberg
Aug 12, 2014, 6:51:04 PM8/12/14
to puppe...@googlegroups.com
On 2014-12-08 19:47, Luke Kanies wrote:
> On Aug 5, 2014, at 6:41 PM, Reid Vandewiele <re...@puppetlabs.com
> On Aug 5, 2014, at 6:41 PM, Reid Vandewiele <re...@puppetlabs.com
> <mailto:re...@puppetlabs.com>> wrote:
>
>> Visual review, for convenience:
>>
>> file { $title: * => $attributes; }
>> file { $title: => $attributes; }
>> file { $title: ($attributes); }
>> file { $title: attribute_defaults => $attributes; }
>> file { $title: attribute_hash => $attributes; }
>
> Sorry for joining the party late; I’m clearly not quite on the same
> email cadence. :/
>
> I think there are some other interesting middle grounds. In particular,
> we could add explicit merge operations, like other languages have for
> hashes:
>
> # Make some functions for merging a hash into a resource
> file { $title: mode => 644 }.merge($attributes) #destructively merge
> file { $title: mode => 644 }.default($attributes) #prefer specified
> attributes
>
Yes sure, but we have earlier frowned on mutating functions.
>
>> Visual review, for convenience:
>>
>> file { $title: * => $attributes; }
>> file { $title: => $attributes; }
>> file { $title: ($attributes); }
>> file { $title: attribute_defaults => $attributes; }
>> file { $title: attribute_hash => $attributes; }
>
> Sorry for joining the party late; I’m clearly not quite on the same
> email cadence. :/
>
> I think there are some other interesting middle grounds. In particular,
> we could add explicit merge operations, like other languages have for
> hashes:
>
> # Make some functions for merging a hash into a resource
> file { $title: mode => 644 }.merge($attributes) #destructively merge
> file { $title: mode => 644 }.default($attributes) #prefer specified
> attributes
>
The syntax that you used as an example will almost work, but the
resource expression has such low precedence that it cannot be directly
combined with a '.', if you put it in parentheses it will work though.
(file {$title: mode => '0644'}).merge($attributes)
You can write such a function now and use with the future parser (in the
version that is to be released in 3.7. In 3.6 future parser the syntax
would more complex since resource creation is not an expression (cannot
be assigned to anything etc. not without jumping through a hoop).
Also, a resource creation can create multiple resources, so the produced
value can be an array of resources, the function would need to take that
into account (apply the hash to all).
Basically, this moves the Resource Override expression in the language
to be a function that operates on a resource reference, or an array of
resource references. (BTW the Resource Override expression is the source
of many grammatical problems, and I would love to get rid of it).
> # Alternative: Make operators for merging
> file { $title: mode => 644 } & $attributes # Or something like that
>
as well. Again, they have to be parenthesized.
(file {$title: mode => '0644' }) + $attributes
This would be simple to implement.
> In other words, get rid of the whole concept of explicitly specifying
> defaults, and also get rid of anything inside the curlies, and just
> modify the resource in place with a function or operator.
>
> In my head, the bitwise operators seemed like an obvious operator set to
> use; in practice, it looks, um, questionable at best. At the least, any
> of the above statements would be expected to affect the resource, and
> it’s tough to say that’s obvious from the syntax.
>
(file {$title: mode => '0644' }) << $attributes
> I do like the explicitness of it, though.
>
inconsistent. By using a collector you can override any resource value,
but you are not allowed to override with the resource override
expression (only set unset values).
We later have to tackle the requirement that it should be possible to
restate the same resource (and possibly amend to it) from multiple
sources, and we do have to have to find a different approach. Only
mutability leads to wild west - however managed to run last gets to set
the values. (In order to support this, the catalog model needs to change
as it can currently only track one containment). This is work planned to
start after Puppet 4.0 BTW.
- henrik
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/69F21637-06EA-428A-AA02-982777591D4F%40puppetlabs.com
> <https://groups.google.com/d/msgid/puppet-dev/69F21637-06EA-428A-AA02-982777591D4F%40puppetlabs.com?utm_medium=email&utm_source=footer>.
Luke Kanies
Aug 12, 2014, 10:39:20 PM8/12/14
to puppe...@googlegroups.com
> On Aug 12, 2014, at 3:51 PM, Henrik Lindberg <henrik....@cloudsmith.com> wrote:
>
>> On 2014-12-08 19:47, Luke Kanies wrote:
>> On Aug 5, 2014, at 6:41 PM, Reid Vandewiele <re...@puppetlabs.com
>> <mailto:re...@puppetlabs.com>> wrote:
>>
>>> Visual review, for convenience:
>>>
>>> file { $title: * => $attributes; }
>>> file { $title: => $attributes; }
>>> file { $title: ($attributes); }
>>> file { $title: attribute_defaults => $attributes; }
>>> file { $title: attribute_hash => $attributes; }
>>
>> Sorry for joining the party late; I'm clearly not quite on the same
>> email cadence. :/
>>
>> I think there are some other interesting middle grounds. In particular,
>> we could add explicit merge operations, like other languages have for
>> hashes:
>>
>> # Make some functions for merging a hash into a resource
>> file { $title: mode => 644 }.merge($attributes) #destructively merge
>> file { $title: mode => 644 }.default($attributes) #prefer specified
>> attributes
> Yes sure, but we have earlier frowned on mutating functions.
I'm not specifically saying this is the right answer, but if your goal
>
>> On 2014-12-08 19:47, Luke Kanies wrote:
>> On Aug 5, 2014, at 6:41 PM, Reid Vandewiele <re...@puppetlabs.com
>> <mailto:re...@puppetlabs.com>> wrote:
>>
>>> Visual review, for convenience:
>>>
>>> file { $title: * => $attributes; }
>>> file { $title: => $attributes; }
>>> file { $title: ($attributes); }
>>> file { $title: attribute_defaults => $attributes; }
>>> file { $title: attribute_hash => $attributes; }
>>
>> Sorry for joining the party late; I'm clearly not quite on the same
>> email cadence. :/
>>
>> I think there are some other interesting middle grounds. In particular,
>> we could add explicit merge operations, like other languages have for
>> hashes:
>>
>> # Make some functions for merging a hash into a resource
>> file { $title: mode => 644 }.merge($attributes) #destructively merge
>> file { $title: mode => 644 }.default($attributes) #prefer specified
>> attributes
> Yes sure, but we have earlier frowned on mutating functions.
is to be more explicit and clear, and to add syntax for this case and
other data manipulation needs, then I think these are in a better
direction than the other proposed syntaxes. Again, directionally
right, even if the specifics aren't exact.
> The syntax that you used as an example will almost work, but the resource expression has such low precedence that it cannot be directly combined with a '.', if you put it in parentheses it will work though.
>
> (file {$title: mode => '0644'}).merge($attributes)
with minimal/no cost?
> You can write such a function now and use with the future parser (in the version that is to be released in 3.7. In 3.6 future parser the syntax would more complex since resource creation is not an expression (cannot be assigned to anything etc. not without jumping through a hoop).
>
> Also, a resource creation can create multiple resources, so the produced value can be an array of resources, the function would need to take that into account (apply the hash to all).
>
> Basically, this moves the Resource Override expression in the language to be a function that operates on a resource reference, or an array of resource references. (BTW the Resource Override expression is the source of many grammatical problems, and I would love to get rid of it).
If we went this kind of route, we'd want to find a way to not require
parens, IMO. Although maybe that would be ugly enough that it would
discourage use, which might be the right behavior. :)
>> # Alternative: Make operators for merging
>> file { $title: mode => 644 } & $attributes # Or something like that
> We already use + for merging, so that could be used to merge resources as well. Again, they have to be parenthesized.
>
> (file {$title: mode => '0644' }) + $attributes
>
> This would be simple to implement.
And note that I'm far more concerned about simple to use than simple
to implement. If we have to do 10x the engineering but our users have
to do 1/10th the thinking to use it, then it's easily worth the
investment.
>> In other words, get rid of the whole concept of explicitly specifying
>> defaults, and also get rid of anything inside the curlies, and just
>> modify the resource in place with a function or operator.
>>
>> In my head, the bitwise operators seemed like an obvious operator set to
>> use; in practice, it looks, um, questionable at best. At the least, any
>> of the above statements would be expected to affect the resource, and
>> it's tough to say that's obvious from the syntax.
>
> i.e. append these attributes:
>
> (file {$title: mode => '0644' }) << $attributes
parser, or if it would be too confusing with the spaceship operator.
Again, parens make the whole thing not so great. And you need a way
to differentiate between the "replace if they're there" and "only add
these if the param isn't set" cases.
>> I do like the explicitness of it, though.
>
> The biggest issue is the immutability aspect. The language is currently inconsistent. By using a collector you can override any resource value, but you are not allowed to override with the resource override expression (only set unset values).
>
> We later have to tackle the requirement that it should be possible to restate the same resource (and possibly amend to it) from multiple sources, and we do have to have to find a different approach. Only mutability leads to wild west - however managed to run last gets to set the values. (In order to support this, the catalog model needs to change as it can currently only track one containment). This is work planned to start after Puppet 4.0 BTW.
we should be trying to fix.
I think the problems in Puppet's language stem more from inconsistency
than from incompleteness[1]. You've already done a lot of good work
to improve that, and I think this is another area where we could make
smaller improvements in completeness and much bigger improvements in
consistency, which would solve nearly everyone's problems by making a
cleaner, clearer design that was a bit more powerful but not too much.
1 - Note that just about my favorite book in the world is Gödel Escher
Bach, so I can talk about this topic for far longer than anyone else
cares. I might be making total gibberish here.
Reply all
Reply to author
Forward
0 new messages