Tags and dependencies

[zerozer...@gmail.com]

Hi,

I'm using tags in resource defaults to define resource ordering inside a single class, preventing at the same time dependency cycles with other classes, like this:

class myname::myclass {

  # defaults

  User       { tag => "this_class" }

  File       { tag => "this_class" }

  Service    { tag => "this_class" }

  # dependencies

  User       <| tag == "this_class" |> ->

  File       <| tag == "this_class" |> ->

  Service    <| tag == "this_class" |>

  # resources, many of each type

  user {

    ...

  }

  file {

    ...

  }

  service {

    ...

  }

}

But this doesn't seem to work: I get errors, and by looking at the report I can see that the errors are due to dependencies not being respected, e.g. some service resources are applied before some file resources and so on.

Am I missing anything?

BTW, I was supposing there are no problems with "nested" dependencies, right?

I mean, the class itself is part of a dependency chain together with other classes, and the chain works fine. But I suppose there is no problem with creating dependencies contained inside a class which is part of the higher chain.

Thanks.

Marco

[jcbollinger]

On Wednesday, February 12, 2014 9:01:40 AM UTC-6, zerozer...@gmail.com wrote:

Hi,

I'm using tags in resource defaults to define resource ordering inside a single class, preventing at the same time dependency cycles with other classes, like this:

class myname::myclass {

  # defaults

  User       { tag => "this_class" }

  File       { tag => "this_class" }

  Service    { tag => "this_class" }

  # dependencies

  User       <| tag == "this_class" |> ->

  File       <| tag == "this_class" |> ->

  Service    <| tag == "this_class" |>

  # resources, many of each type

  user {

    ...

  }

  file {

    ...

  }

  service {

    ...

  }

}

But this doesn't seem to work: I get errors, and by looking at the report I can see that the errors are due to dependencies not being respected, e.g. some service resources are applied before some file resources and so on.

Am I missing anything?

I think so.  This looks related to https://tickets.puppetlabs.com/browse/PUP-1045.  Moreover, resources get automatic tags that may support what you're after even easier than what you're trying to do.  For example, this model might work:

class myname::myclass {

  # no resource defaults needed

  # dependencies

  User       <| tag == "myname::myclass" |> ->

  File       <| tag == "myname::myclass" |> ->

  Service    <| tag == "myname::myclass" |>

  ...

}

 

BTW, I was supposing there are no problems with "nested" dependencies, right?

I mean, the class itself is part of a dependency chain together with other classes, and the chain works fine. But I suppose there is no problem with creating dependencies contained inside a class which is part of the higher chain.

You are mostly correct.  There is no problem with resources being chained and the class or defined type instance containing them being part of a separate chain.  Resources declared directly by a class or defined type instance are contained in their declaring class or defined type instance.

Classes, however, are not resources, no matter how resource-like Puppet makes them appear.  Classes declared via the 'include' function or the resource-like declaration syntax are not automatically contained.  Where needed, they can be made contained via the new 'contain' function.

Do note, however, that what you're specifically doing seems questionable, in that you are declaring many relationships that you probably don't need.  For example, suppose your class manages both NTP and Puppet.  You may have users 'ntp' and 'puppet', files '/etc/ntp.conf' and '/etc/puppet/puppet.conf', and services 'ntp' and 'puppet'.  Why do you need a relationship between User['ntp'] and File['/etc/puppet/puppet.conf']?

Unneeded relationships invite trouble.  They make resource cycles more likely, and they make resource failures more impactful than they need to be.  They may also make your catalogs larger and your Puppet runtimes longer.


John

[zerozer...@gmail.com]

On Thursday, February 13, 2014 4:23:27 PM UTC+1, jcbollinger wrote:

I think so.  This looks related to https://tickets.puppetlabs.com/browse/PUP-1045.

Uhm… thanks! So the problem is that defaults only apply when the specified attributes are empty for the affected resources; but some tags are always present by default, so you cannot assign default tags.

…actually, I have to say this looks like an inconsistency, and it should be ideally fixed, like the comment on that issue states.

Tags are special attributes after all, you cannot replace them (right?), your tags are always appended to the already existing ones instead of replacing them, so tags in resource defaults should have the same behavior.

Moreover, resources get automatic tags that may support what you're after even easier than what you're trying to do.

Yes, I discovered this while searching for solutions to my problem.

I guess I will need to try this way now.

Do note, however, that what you're specifically doing seems questionable, in that you are declaring many relationships that you probably don't need.  For example, suppose your class manages both NTP and Puppet.  You may have users 'ntp' and 'puppet', files '/etc/ntp.conf' and '/etc/puppet/puppet.conf', and services 'ntp' and 'puppet'.  Why do you need a relationship between User['ntp'] and File['/etc/puppet/puppet.conf']?

Unneeded relationships invite trouble.  They make resource cycles more likely, and they make resource failures more impactful than they need to be.  They may also make your catalogs larger and your Puppet runtimes longer.

I know this has some logic problems, but I wanted to keep manifests clean.

For example, I have some files which need to be put in place before the related services are started, so instead of creating lots of file->service pairs I collected all services in a single resource in condensed form, which will be applied after all files are ready.

Thanks.

Marco

[zerozer...@gmail.com]

On Thursday, February 13, 2014 4:23:27 PM UTC+1, jcbollinger wrote:

I think so.  This looks related to https://tickets.puppetlabs.com/browse/PUP-1045.  Moreover, resources get automatic tags that may support what you're after even easier than what you're trying to do.  For example, this model might work:

class myname::myclass {

  # no resource defaults needed

  # dependencies

  User       <| tag == "myname::myclass" |> ->

  File       <| tag == "myname::myclass" |> ->

  Service    <| tag == "myname::myclass" |>

  ...

}

Hi again,

I'm trying this but for some reason it isn't working.

For example, if I have a File resource to which I assigned a custom "post_exec" tag because it needs to be applied after any Exec resource, and the following code works fine:

class myname::myclass {

  Exec <| |> ->

  File <| (tag == "post_exec") |>

  ...

}

But if I want to restrict the dependency to my specific class, to prevent problems, the following code fails, the dependency is not created (I checked the difference with puppet agent --test --graph):

class myname::myclass {

  Exec <| (tag == "myname::myclass") |> ->

  File <| (tag == "myname::myclass") and (tag == "post_exec") |>

  ...

}

Now that I think about it… http://docs.puppetlabs.com/puppet/latest/reference/lang_collectors.html tells that "Collectors can only search on attributes which are present in the manifests and cannot read the state of the target system".

I would not expect the class name tag to only be part of the target system, but the automatically created tag is not explicitly present in the manifest, either. So maybe that's why my code does not work?

Can resource collectors access automatically created tags in their search expressions?

Thanks.

Marco

[Felix Frank]

On 02/14/2014 03:52 PM, zerozer...@gmail.com wrote:
> File <| (tag == "myname::myclass") and (tag == "post_exec") |>

While this looks sane, this syntax is not yet supported. Your collectors
must use trivial expressions of the form "a == b".

Sorry, cheers,
Felix

[zerozer...@gmail.com]

Sorry, what do you mean? I'm using two trivial expressions AFAICT.

Or are you referring to the "and"ing of two search expressions?

According to http://docs.puppetlabs.com/puppet/latest/reference/lang_collectors.html#and it should be supported:

------------------------

and

Both operands must be valid search expressions.

For a given resource, this operator will match if both of the operands would match for that resource.

------------------------

Thanks.

Marco

[Felix Frank]

Argh, umm, never mind then - guess I'm a 2.7.x fossil after all ><'

[jcbollinger]

On Friday, February 14, 2014 9:18:21 AM UTC-6, zerozer...@gmail.com wrote:

On Friday, February 14, 2014 3:55:29 PM UTC+1, Felix.Frank wrote:

On 02/14/2014 03:52 PM, zerozer...@gmail.com wrote:
>   File <| (tag == "myname::myclass") and (tag == "post_exec") |>

While this looks sane, this syntax is not yet supported. Your collectors
must use trivial expressions of the form "a == b".

Sorry, what do you mean? I'm using two trivial expressions AFAICT.

Or are you referring to the "and"ing of two search expressions?

Yes, I believe the combination of two simple expressions into a complex one via the "and" operator is what Felix was referring to.  Despite the docs, it might be worthwhile to play around a bit to test the outlines of the problem:

• Does it really work at all (for your version of the master)?  The docs are fairly reliable, but if you happen not to be on the latest Puppet then they might not apply.  Also, Puppet occasionally suffers regressions (which typically are fixed pretty quickly once PL is made aware of them).
• Is the failure related specifically to the 'tag' property, which is a bit special?
• Is the failure related to having two conditions on the same resource parameter?
  

Or we could return to my original remark that what you're doing is a bit questionable.  Whether your code really ought to work notwithstanding, your manifest set would be more robust if you wrapped all the managed resources related to each piece of software into its own class or defined-type instance.  If you have a lot of services and your declarations are consistent in form, then using a common defined type and one instance per (user, file, service) triple would make your manifests even cleaner, and allow you to be more precise with relationships, too.


John

[zerozer...@gmail.com]

On Monday, February 17, 2014 3:49:30 PM UTC+1, jcbollinger wrote:

 

Yes, I believe the combination of two simple expressions into a complex one via the "and" operator is what Felix was referring to.  Despite the docs, it might be worthwhile to play around a bit to test the outlines of the problem:

• Does it really work at all (for your version of the master)?  The docs are fairly reliable, but if you happen not to be on the latest Puppet then they might not apply.  Also, Puppet occasionally suffers regressions (which typically are fixed pretty quickly once PL is made aware of them).
• Is the failure related specifically to the 'tag' property, which is a bit special?
• Is the failure related to having two conditions on the same resource parameter?
  

I'll be happy to test this but I don't have much time to do it these days, unfortunately.

Anyway both master and agent run puppet 3.4.2.

Or we could return to my original remark that what you're doing is a bit questionable.  Whether your code really ought to work notwithstanding, your manifest set would be more robust if you wrapped all the managed resources related to each piece of software into its own class or defined-type instance.  If you have a lot of services and your declarations are consistent in form, then using a common defined type and one instance per (user, file, service) triple would make your manifests even cleaner, and allow you to be more precise with relationships, too.

Indeed, I got rid of those dependencies between types in the meantime and I'm trying to maintain dependencies only between specific resources.

Your suggestion is interesting, maybe a bit overkill for my current needs but I'll keep it in mind for the future.

Thanks.

Marco