Best practices for ensure property values

[Jesse Hathaway]

In my discussion with @adrienthebo on my pull request: https://github.com/puppetlabs/puppet/pull/2309 I raised the question of what should be the allowable or suggested values for the ensure property?

My pull request separated out the hold state into a separate property:

before:

package { 'foo':
  ensure => held

}

after:

package { 'foo':

  ensure => '0.4',

  hold => false

}

@adrienthebo suggested I separate out the version instead:

package { 'foo':

  ensure => held,

  version => '0.4',

}

@adrienthebo's suggestion is certianly doable, my question is whether the puppet community has come to a consensus on what values the ensure property should have?

In general my experience has been that adding additional values to ensure for resources makes using a resource less intuitive. For instance I find the file resource has a confusing interface because of the way it abuses the ensure property:

file {'/foo':

  ensure => file,

}

file {'/foo':

  ensure => directory,

}

I would rather have explicit resources:

directory {'/foo':

  ensure => present

}

What are the communities thoughts?

-Jesse Hathaway

[John Bollinger]

On Wednesday, March 26, 2014 1:12:33 PM UTC-5, Jesse Hathaway wrote:

In my discussion with @adrienthebo on my pull request: https://github.com/puppetlabs/puppet/pull/2309 I raised the question of what should be the allowable or suggested values for the ensure property?

My pull request separated out the hold state into a separate property:

before:

package { 'foo':
  ensure => held

}

after:

package { 'foo':

  ensure => '0.4',

  hold => false

}

@adrienthebo suggested I separate out the version instead:

package { 'foo':

  ensure => held,

  version => '0.4',

}

@adrienthebo's suggestion is certianly doable, my question is whether the puppet community has come to a consensus on what values the ensure property should have?

Consensus?  Are we talking about the same community?

 

In general my experience has been that adding additional values to ensure for resources makes using a resource less intuitive. For instance I find the file resource has a confusing interface because of the way it abuses the ensure property:

file {'/foo':

  ensure => file,

}

file {'/foo':

  ensure => directory,

}

I would rather have explicit resources:

directory {'/foo':

  ensure => present

}

What are the communities thoughts?

I can only speak for myself, but I don't think there's a clear-cut general rule for what 'ensure' values should be supported by different resource types (among those that support such a parameter at all).  I think the fact that most ensurable types have only 'present' and 'absent' as supported 'ensure' values came about more by happenstance than by design. There is at least one ensurable type -- Service -- that seems reasonably well designed to me with an entirely different set of ensure values.

The language reference describes 'ensure' as managing "the most fundamental aspect of the resource on the target system."  For many types, simple presence or absence fits that description, but I don't necessarily call it abuse to support finer gradations of "present", as File does.  On the other hand, there are types, such as Mount, that do seem to commingle primary and secondary attributes in their collection of ensure values.  That presents both problems on several levels.

Additionally, there are types for which the standard term 'present' is distinctly less meaningful than an alternative term.  Going back to Mount for an example, the ensure value 'defined' is much more intuitive to me for that type than the equivalent 'present'.  That would be the case even if the the mount state (mounted / unmounted) were pulled out of ensure into its own property, as it probably should be.

With respect to Packages, I am inclined to say that whether a package is 'held' should be managed as a separate property because, as I understand it, it models a flag that can be twiddled for an installed package.  That's not nearly so fundamental as the package version -- different versions of the "same" package are in many ways entirely different packages.  Perhaps the package version might be better modeled as a separate attribute, too, but that's much less clear cut.

File is another interesting study.  As I wrote earlier, I disagree that it is an abuse for that type to support ensure values identifying the type of File.  The File type doe shave its share of problems, but to a large extent I think they spring from the ubiquity of files in the UNIX design.  In the past I have discussed the possibility of a separate Directory type, I am somewhat dubious of that idea's practical merit.  I would be willing to entertain the idea of describing the type of file with a separate property, but even there I'm not persuaded.

Where File does abuse 'ensure', though, is with its provision for using it to identify symbolic link targets.  Thankfully, that usage is now discouraged.


John

[Daniele Sluijters]

Hey,

As far as package is concerned I prefer Adrien's suggestion of ensure => held, version => 0.4. This is entirely personal but I think ensure is a good place to model the 'state' of a package and when it comes to apt that state is either held, removed, purged or installed. I would go as far as to say that 'ensure => latest' should not exist, it should be 'ensure => installed with version => latest'.

If I think about it in English I'd say "I want the latest version of this package installed" which is why the mapping "version => latest" makes more sense to me as does an 'ensure => installed'. I would also say something like "I want nginx held at 1.4.7" which fairly easily translates to "ensure => held, version => 1.4.7". This specific use-case of holding a version isn't even possible with the current package type because of how the apt provider works, it will hold the current installed version, you can't request it to install a specific version and hold that one except for messing with apt policies.

I have no idea how this translates to other package managers though.

As far as 'ensure => directory' on a file goes I don't really mind that syntax, in my mind a directory is just a special file, much like a symlink is or a socket. A separate directory type might be interesting if it has extra magical powers that the file type doesn't, like having a flag that implies `mkdir -p` instead of `mkdir` for example.

-- 

Daniele Sluijters

[Jesse Hathaway]

The language reference describes 'ensure' as managing "the most fundamental aspect of the resource on the target system."  For many types, simple presence or absence fits that description, but I don't necessarily call it abuse to support finer gradations of "present", as File does.  On the other hand, there are types, such as Mount, that do seem to commingle primary and secondary attributes in their collection of ensure values.  That presents both problems on several levels.

Thanks for pointing out this definition. I think asking whether an ensure parameter models a fundamental aspect of a resource is a helpful litmus test for what values an ensure parameter should have for a resource.

 

Additionally, there are types for which the standard term 'present' is distinctly less meaningful than an alternative term.  Going back to Mount for an example, the ensure value 'defined' is much more intuitive to me for that type than the equivalent 'present'.  That would be the case even if the the mount state (mounted / unmounted) were pulled out of ensure into its own property, as it probably should be.

With respect to Packages, I am inclined to say that whether a package is 'held' should be managed as a separate property because, as I understand it, it models a flag that can be twiddled for an installed package.  That's not nearly so fundamental as the package version -- different versions of the "same" package are in many ways entirely different packages.  Perhaps the package version might be better modeled as a separate attribute, too, but that's much less clear cut.

I have never liked how the syntax reads, e.g. ensure => '4.1', but I agree with you that it passes the litmus test of being a fundamental aspect.

 

File is another interesting study.  As I wrote earlier, I disagree that it is an abuse for that type to support ensure values identifying the type of File.  The File type doe shave its share of problems, but to a large extent I think they spring from the ubiquity of files in the UNIX design.  In the past I have discussed the possibility of a separate Directory type, I am somewhat dubious of that idea's practical merit.  I would be willing to entertain the idea of describing the type of file with a separate property, but even there I'm not persuaded.

My problem with the file type is that though you might be able to access directories and symbolic links as files in Unix, they model different abstractions. I think you see evidence of this abstraction in the separate Unix and Windows cli tools, mkdir, ln, mklink. That is why I find it a strange mental model that when I need to create a directory or a symbolic link I need to use the file type.

-Jesse

[Andy Parker]

On Thu, Mar 27, 2014 at 12:22 PM, Daniele Sluijters <daniele....@gmail.com> wrote:

Hey,

As far as package is concerned I prefer Adrien's suggestion of ensure => held, version => 0.4. This is entirely personal but I think ensure is a good place to model the 'state' of a package and when it comes to apt that state is either held, removed, purged or installed. I would go as far as to say that 'ensure => latest' should not exist, it should be 'ensure => installed with version => latest'.

If I think about it in English I'd say "I want the latest version of this package installed" which is why the mapping "version => latest" makes more sense to me as does an 'ensure => installed'. I would also say something like "I want nginx held at 1.4.7" which fairly easily translates to "ensure => held, version => 1.4.7". This specific use-case of holding a version isn't even possible with the current package type because of how the apt provider works, it will hold the current installed version, you can't request it to install a specific version and hold that one except for messing with apt policies.

I have no idea how this translates to other package managers though.

I think the crux of the matter is how well the proposal would translate to other package managers. I think that the proposed separation of ensure and version makes sense, but it also opens up a few edge cases that having all of it in ensure doesn't. For instance, what should be expected to happen if the package is "ensure => held, version => 1.4.3" but version 2.1 is actually installed? I expect the answer is to downgrade to 1.4.3 and do whatever the package manager can do for holding. But is that a universal enough capability?

 

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/feb80058-5a45-46f0-8b05-cb86b54b645c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Parker

an...@puppetlabs.com

Freenode: zaphod42

Twitter: @aparker42

Software Developer

Join us at PuppetConf 2014, September 23-24 in San Francisco - http://puppetconf.com

[Henrik Lindberg]

My commments on the topic of ensure property values are not specific to
any of the particular previous comments, so I am top posting.

The boundary between which part of the resource's life-cycle that should
be described by the ensure parameter, and other parameters/properties is
very blurry. This since *all* of them are used to describe the desired
state of the resource.

It is possible to model this as a structured state machine. The basic
life cycle states are: exists/present, and non-existing/not-present. For
some types it may be more natural to talk about their
existence/non-existence in terms of other words - i.e.
"defined"/"undefined" (but it is basically the same states). When the
resource exists, it can have sub-states, and some of those are in
parallel (it can be big & red, or small & blue), some states are
mutually exclusive, and some states are best modeled as additional sub
states (e.g. exists/configured/active as opposed to
exists/configured/inactive).

While it is fine to use only ensure to describe the desired state out of
all of the available states (as long as they are uniquely identified)
since such a state also specifies all super state (e.g. 'active'
implies, 'configured' and 'exists').

The problem here is one of validation and documentation; how can code
declare the rules, how can they be documented so that it is clear what a
particular value means, and what additional properties then apply.

Secondly, when scalar values are used as value of ensure, this implies a
"type to state coersion", i.e. if a string is given it implies "exists
of given version". This required yet more mental capacity.

While it is easy to read the code; you can guess what "ensure => '4.1'"
means, it is harder to construct given an arbitrary type. Tooling (such
as Geppetto) have a hard time helping a user since the information
regarding what can be used is buried in general Ruby logic. Tools like
Geppetto would benefit greatly by having the ensure parameter be typed
by a state machine.

I got some good things to think about from this topic when we are now
trying to design a better resource type API for Puppet 4.

Back to best practices. I would suggest only using what comes across as
a named state in ensure, and have things like "version" be a separate
property even if it is slightly more to type.

Regards
- henrik

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com

> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/CANSNSoV7_9L0O15D15kdMmYkxtpjSYNeXWdcyrkOO6m0jV0kPA%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CANSNSoV7_9L0O15D15kdMmYkxtpjSYNeXWdcyrkOO6m0jV0kPA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Felix Frank]

On 03/28/2014 04:01 PM, Jesse Hathaway wrote:
> Thanks for pointing out this definition. I think asking whether an
> ensure parameter models a fundamental aspect of a resource is a
> helpful litmus test for what values an ensure parameter should have
> for a resource.

Nitpick: Ensure is universally a property, not a parameter, i.e. it is
quite capable to go out of sync, which will prompt a reaction from
Puppet. This in and of itself can be an indicator as to what values may
make (no) sense.

> My problem with the file type is that though you might
> be able to access directories and symbolic links as files in Unix,
> they model different abstractions. I think you see evidence of this
> abstraction in the separate Unix and Windows cli tools, mkdir, ln,
> mklink. That is why I find it a strange mental model that when I need
> to create a directory
> or a symbolic link I need to use the file type.

We're getting side-tracked ;) but here's why those should really be the
same to Puppet: What the catalog refers to as a 'file' is actually a
file system entry.

If you allow the resources file { "/tmp": } and directory { "/tmp": } to
coexists, you are bound to face trouble. Since both represent the same
file system entry, puppet should handle them with exactly one resource.
It's a sane design in the given domain.

On 03/28/2014 05:00 PM, Andy Parker wrote:
> I have no idea how this translates to other package managers though.
>
>
> I think the crux of the matter is how well the proposal would translate
> to other package managers. I think that the proposed separation of
> ensure and version makes sense, but it also opens up a few edge cases
> that having all of it in ensure doesn't. For instance, what should be
> expected to happen if the package is "ensure => held, version => 1.4.3"
> but version 2.1 is actually installed? I expect the answer is to
> downgrade to 1.4.3 and do whatever the package manager can do for
> holding. But is that a universal enough capability?

I agree with the sentiment that this is indeed the crux.

However, since the semantics in the described scenario are so
intuitively clear, I would actually expect Puppet to fail the resource
if my package manager is incapable of performing the necessary steps.

[Jesse Hathaway]

On Fri, Mar 28, 2014 at 11:12 AM, Felix Frank <felix...@alumni.tu-berlin.de> wrote:

We're getting side-tracked ;) but here's why those should really be the
same to Puppet: What the catalog refers to as a 'file' is actually a
file system entry.

True, but that is not how you think about them when you are creating them. You don't think I need to create a file system entry that is a directory. You think I need to create a directory.

 

If you allow the resources file { "/tmp": } and directory { "/tmp": } to
coexists, you are bound to face trouble. Since both represent the same
file system entry, puppet should handle them with exactly one resource.
It's a sane design in the given domain.

That is a problem puppet could alert on, or puppet could just let the managed OS tell whether that state is permissible.

[Felix Frank]

Hi,

On 03/28/2014 08:14 PM, Jesse Hathaway wrote:

If you allow the resources file { "/tmp": } and directory { "/tmp": } to
coexists, you are bound to face trouble. Since both represent the same
file system entry, puppet should handle them with exactly one resource.
It's a sane design in the given domain.

That is a problem puppet could alert on, or puppet could just let the managed OS tell whether that state is permissible.

I may be missing something, but that can never be a consistent single state on any platform.
A path will always lead to a file *or* a directory *or* a symlink.

Allowing a directory { } syntax might be conceivable if directory and link were basically aliases for the file resource type. They would munge the ensure values differently.

I don't think that would add much value though, because behind the scenes, they would map back to the file type. To the user, this would potentially be very confusing. (Also, I sense much opportunity for weird bugs in the implementation.)

Regards,
Felix

[Jesse Hathaway]

On Sat, Mar 29, 2014 at 10:05 AM, Felix Frank <Felix...@alumni.tu-berlin.de> wrote:

I may be missing something, but that can never be a consistent single state on any platform.
A path will always lead to a file *or* a directory *or* a symlink.

true, I think I would be somewhat more comfortable with the single resource if it was named better, e.g. path or something similar

[Jesse Hathaway]

I think the discussion has petered out, I propose the following.

Given that held is arguably not an essential state of a package nor version, we should move to the following package declaration,

package { 'foo':

  ensure => installed,

  version => '1.1',

  hold => true,

}

thoughts?, objections?

[John Bollinger]

I'm generally ok with that, provided that the "held" property is spelled correctly (property names should normally be adjectives or nouns).  However, the ensure => '<version>' idiom seems widely enough used that it should not be dropped without a deprecation / warning period.  Perhaps the same is true of 'held'.  I don't anyway think that a breaking change such as this can be released before the next major version.


John

[Trevor Vaughan]

I have to say that I like 'hold' better than 'held' based on the way I would actually read this.

This is package 'foo'.

I want to ensure that it is installed.

I want the version to be '1.1'.

I want to hold it fast.

I want it held fast works as well, but just sits funny with me somehow.

Not much of a contribution to the conversation, but there you go....

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/b7fb208b-9990-4aec-88dd-463c1551c5ee%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[Daniele Sluijters]

If we go down this road, how does 'purged' fit in. Should we have purge => true, or ensure => 'purged'?

On Wednesday, 26 March 2014 19:12:33 UTC+1, Jesse Hathaway wrote:

[Trevor Vaughan]

I like hold and purge as separate parameters for readability (which translates to being easier to parse with external scripts if necessary).

Trevor

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/8af42114-8ec1-4d85-9000-096031d98e9a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Felix Frank]

On 04/01/2014 01:44 AM, Trevor Vaughan wrote:
> I have to say that I like 'hold' better than 'held' based on the way I
> would actually read this.

Right, and I don't agree about John's point about property names. There
are counter-examples that work very well like service/enable,
file/purge, file/recurse etc. The inconsistency with yumrepo/enabled is
unfortunate.

When I read those, I think 'imperative', which really is what the
manifest represents. Ensure this! Hold that! :-)

On 04/01/2014 03:20 PM, Trevor Vaughan wrote:
> I like hold and purge as separate parameters for readability (which
> translates to being easier to parse with external scripts if necessary).
>
> Trevor
>
>
> On Tue, Apr 1, 2014 at 9:09 AM, Daniele Sluijters
> <daniele....@gmail.com <mailto:daniele....@gmail.com>> wrote:
>
> If we go down this road, how does 'purged' fit in. Should we have
> purge => true, or ensure => 'purged'?

Hmm, no.

Seeing as 'purged' in this context is an installation state competing
with installed or removed, it should stay in the ensure property, which
is what manages installation states. E.g., this makes little sense:

package { 'foo': ensure => 'installed', purge => true }

Cheers,
Felix

[Daniele Sluijters]

By that same logic held/hold should also stay in ensure but that's not feasible because you can't hold at a specific version then.

[Felix Frank]

Hi,

well, no and no. The "hold" state is (to my knowledge and at least
pertaining to apt/dpkg) independent of the installation status. I can
remove a held package, it just won't update to a newer version.

Even if we were to keep 'held' as an ensure value, this would even still
work thanks to the introduction of the version property. I don't think
it would be a good idea though.

Regards,
Felix

On 04/01/2014 04:39 PM, Daniele Sluijters wrote:
> By that same logic held/hold should also stay in ensure but that's not
> feasible because you can't hold at a specific version then.

[John Bollinger]

On Tuesday, April 1, 2014 8:25:21 AM UTC-5, Felix Frank wrote:

On 04/01/2014 01:44 AM, Trevor Vaughan wrote:
> I have to say that I like 'hold' better than 'held' based on the way I
> would actually read this.

Right, and I don't agree about John's point about property names. There
are counter-examples that work very well like service/enable,
file/purge, file/recurse etc. The inconsistency with yumrepo/enabled is
unfortunate.

Puppet resources and resource properties describe aspects of the target system's configuration.  Virtually all such aspects are things or attributes of things -- nouns or adjectives -- and it best models the target to use labels that are consistent with that nature.  That there are historic examples that do not comply with that principle is unfortunate, but not a good reason to reject the principle for future development.

Moreover, of the three examples presented, two -- file/purge and file/recurse -- are only parameters, not properties.  The same principle does not necessarily apply to parameters, because those may in some cases describe something the puppet agent is supposed to do (a verb), or how it is supposed to do it (an adverb).

 

When I read those, I think 'imperative', which really is what the
manifest represents. Ensure this! Hold that! :-)

But that's just the thing: to whom is such an imperative directed?  If it is directed to the Puppet agent then a verb is fine, but in that case the parameter you are dealing with is likely not a property.  ('Ensure' is a special case that we can discuss further if desired, but for the moment just observe that it is indeed an instruction to Puppet, hence a verb is reasonable.)  On the other hand, consider held/hold: puppet does not have the ability to hold packages, so it is sloppy to name a resource property as if it did.  What it really does for the held/hold property is configure the package management system to hold the package.  The state of such a package is "held", not "hold".

That distinction is colorable, of course.  For instance, one could say that "hold" is an instruction to Puppet, telling it to put the package in the "held" state (kinda roundabout, but whatever).  Consider, however: is that really what you would be telling the agent?  In fact not.  What you would be telling it is to check whether the package is already in the "held" state, and if not then to put it in that state.  That's not very well summed up by 'hold', but perhaps the property could be named "if_not_held_hold".  Or not.

Appropriate naming is important because the names we use influence our thinking about the named things.  We should choose names that promote clear thinking.  So in the end, yes, it's about how the declaration reads.  The "hold" alternative does not necessarily read badly -- that's subjective value judgement -- but it does read wrongly.


John

[John Bollinger]

On Monday, March 31, 2014 6:44:35 PM UTC-5, Trevor Vaughan wrote:

I have to say that I like 'hold' better than 'held' based on the way I would actually read this.

This is package 'foo'.

I want to ensure that it is installed.

"is"

 

I want the version to be '1.1'.

"be"

 

I want to hold it fast.

"hold"?  I daresay not.  You don't want to hold it, you want it held for you.

John

[Andy Parker]

I'm losing track of this conversation. Let's try to bring it in for a landing so that we can move forward with the pull request from Jesse.

Is there a general consensus that this is the correct direction?

  package { 'foo':
    ensure => installed,
    version => '1.1',

    held => true,
  }

I can see either held being part of ensure or not, so I'll leave the final decision to those of you who use puppet every day.

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/88eefe32-2c56-4837-896f-ea0373a92088%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

[Robin Bowes]

Are hashes as property values out of the question?

eg.

ensure => {
  installed => true,
  version => '1.2.3',
  held => true,
  bikeshed => 'green',
}

R.

[Felix Frank]

On 04/01/2014 08:41 PM, Robin Bowes wrote:
> Are hashes as property values out of the question?

I'll go on a limb and answer this with a strong affirmative ;-) Good lord...

As for held vs. hold, I guess John's points have more merit than mine
after all.

I vote to keep held out of ensure. It really is an independent property.

Cheers,
Felix

[Felix Frank]

On 04/02/2014 01:11 AM, Felix Frank wrote:
>> Are hashes as property values out of the question?
> I'll go on a limb and answer this with a strong affirmative ;-) Good lord...

More to the point: Hash values in and of themselves are fine. Not for
ensure though, and especially not in a way that makes ensure more or
less the only property. That would just shift the problem to a meta
level instead of solving it.