problems with dependency handling in puppet and a proposal to get rid of them
Lorenz Schori
This is my first post on this list, so i'm obviously not a long time
puppet user/dev. Nevertheless i want to propose a change to improve the
semantic correctness and reliability of the dependency handling system
of puppet.
I've been working on some puppet modules to cope with chroots, bind
mounts and lvm snapshots under debian when i was bitten by a nasty
problem. The construction of the resources was easy: create two
file-resources for the source-dir and the mountpoint, create a mount-
resource to bind-mount the source-dir to the mountpoint. Obviously the
mount-resource requires the two file-resources. In the other direction
this worked not as expected. because mount-requires the file i either
had problems with mounts still being busy or - much more dangerous -
completely empty source directories because my mountpoint-file forcibly
and recursively remove all the contents under the mountpoint before the
unmount took place. The workaround was to create a define wrapping the
mount and either call it with a "require" (on creation (ensure =>
present)) or with a "before" argument (on deletion (ensure => absent))
to invert the order the resources get handled when the bind mount
should be removed. I've attached a fully working example, so people
using Linux may play around with it.
I also ran into problems like this when working with users and groups,
when puppet attempted to remove the primary group of a user before the
user itself.
Anyway, to cut a long story short i propose to add two new meta
parameters called "above" and "below", where the direction is given by
the state change of a resource. Lets assume that we construct a system
from bottom to top (bottom = bare os, top = magic self healing cluster
of number crunchers calculating the answer to the life, the universe and
everything), so if ensure is something like present, installed, mounted
this will imply that the state of a resource is augmented and thus other
resources below that one must be constructed before and those above
this resource afterward. On the other side if ensure is something like
absent or purge resources above this one must be handled before and the
ones below after.
I hope was not to unclear. I'm sure this would solve much problems i
worked around in my receipts without breaking the current dependency
model or did i miss something crucial?
Looking forward to a interesting discussion
Lorenz
Luke Kanies
> Anyway, to cut a long story short i propose to add two new meta
> parameters called "above" and "below", where the direction is given by
> the state change of a resource. Lets assume that we construct a system
> from bottom to top (bottom = bare os, top = magic self healing cluster
> of number crunchers calculating the answer to the life, the universe
> and
> everything), so if ensure is something like present, installed,
> mounted
> this will imply that the state of a resource is augmented and thus
> other
> resources below that one must be constructed before and those above
> this resource afterward. On the other side if ensure is something like
> absent or purge resources above this one must be handled before and
> the
> ones below after.
>
> I hope was not to unclear. I'm sure this would solve much problems i
> worked around in my receipts without breaking the current dependency
> model or did i miss something crucial?
I actually tried this maybe 1.5 years ago, and it was basically
completely untenable and pissed everyone off.
The crux is: What happens if you need to delete a resource in order
to correctly configure another resource? That is, correct
configuration of a given resource class requires negation of some
resource.
In that case, we would incorrectly set order, which would then break
everything.
The only real way to do something like this is to have class-level
'ensure'-equivalents: We'd need to be able to talk about the whole
service class being present or running or whatever, and then the
'ensure' state of all contained resources would be bit-flipped
depending on what we did with that class-level state.
This is something that I've considered, but it's a drastic change to
Puppet's language, transactional system, and probably more, so I
haven't embarked on it yet. Maybe for 2.0 or something?
--
A Chemical Limerick:
A mosquito cried out in pain:
"A chemist has poisoned my brain!"
The cause of his sorrow
was para-dichloro
diphenyltrichloroethane
-- Dr. D. D. Perrin
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com
Lorenz Schori
Luke Kanies <lu...@madstop.com> wrote:
Ok, first i want to say that i'm very happy with puppet and the way it
works. The require-thing is just one little annoyance which really
bugged me long enough that i decided to look at it closer. So if i
criticize some puppet internals i try to find the most appropriate fair
and clear words. I don't intend to be rough against you luke and all
the other developers who realized this wonderful system.
> [...]
>
> The crux is: What happens if you need to delete a resource in order
> to correctly configure another resource? That is, correct
> configuration of a given resource class requires negation of some
> resource.
>
> In that case, we would incorrectly set order, which would then break
> everything.
I agree, but this is a special case. Let's treat that special and
carefully instead of complicate the standart cases. I argue that it
should be easy to add *and* remove resources which depend on each other
without requiring users to work around dependency-problems for the
ensure => absent case for each and every resource. Additionally with
the "above"/"below" solution "require" and "before" will not be touched
in any way, they continue to behave like before. This two dependency
models may coexist as long as they will not be intermixed heavily -
that's actually a thing which i want to test more in depth.
I've attached POC patches and some test cases to illustrate the
situation. Note the mybind-define. This is one of my most used patterns
and the reason i'm not happy with the dependency system.
> The only real way to do something like this is to have class-level
> 'ensure'-equivalents: We'd need to be able to talk about the whole
> service class being present or running or whatever, and then the
> 'ensure' state of all contained resources would be bit-flipped
> depending on what we did with that class-level state.
I don't thing that this is necessary. I do often something like this:
class apache {
# general defines go here
package { "apache2": ensure => installed }
service { "apache2": enable => false, ensure => stopped }
}
class apache::enable inherits apache {
# config stuff goes here
Service["apache"] { enable => true, ensure => running }
}
class apache::disable inherits apache {
# special clean up stuff (rarely used)
Service["apache"] { enable = false, ensure => stopped }
}
Then i include apache or apache::enable in specialized classes or even
inherit from apache::enable and include them in node definitions. I
think it is better to not try to make puppet smarter than it can be. The
bitflipping approach sounds to me like a very bad solution to a not so
bad problem - at least if i did understand your short description
correctly.
Lorenz
Brice Figureau
> On Fri, 24 Oct 2008 11:48:17 -0500
> Luke Kanies <lu...@madstop.com> wrote:
Please do not attach your patches in your mails but instead use
git-send-email so that they appear in the mail body. This way we can
review and comment easily.
For more information about puppet patch submission, please read:
http://reductivelabs.com/trac/puppet/wiki/DevelopmentLifecycle#submitting-patches
I'm copying here a snippet of your patch that seems to contain a typo:
diff --git a/lib/puppet/type.rb b/lib/puppet/type.rb
index 85f72f7..88b855a 100644
--- a/lib/puppet/type.rb
+++ b/lib/puppet/type.rb
@@ -1533,9 +1533,18 @@ class Type
self.fail "Could not retrieve dependency '%s' of
%s" % [reference, @resource.ref]
end
+ deleting = (@resource[:ensure] == :absent ||
@resource[:ensure] == :purge)
+ if self.class.direction == :inoncreation
+ direction = deleting ? :out : :in
+ elsif self.class.diroction == :inondeletion
^
This looks like a typo---------------+
+ direction = deleting ? :in : :out
+ else
+ direction = self.class.direction
+ end
+
--
Brice Figureau
Days of Wonder
http://www.daysofwonder.com
Lorenz Schori
Brice Figureau <brice-...@daysofwonder.com> wrote:
> [...]
> Please do not attach your patches in your mails but instead use
> git-send-email so that they appear in the mail body. This way we can
> review and comment easily.
>
> For more information about puppet patch submission, please read:
> http://reductivelabs.com/trac/puppet/wiki/DevelopmentLifecycle#submitting-patches
Ok, thanks for link. Like i said before, the patches are POC-state and
just to illustrate the idea and seeking interest. I'm not happy about
the options-mechanism but it seemed to be the cheapest way add the
desired behavior.
> I'm copying here a snippet of your patch that seems to contain a typo:
> [...]
> + if self.class.direction == :inoncreation
> + direction = deleting ? :out : :in
> + elsif self.class.diroction == :inondeletion
> ^
> This looks like a typo---------------+
Yes, this is a typo. I'm just putting together my testing environment,
so i can easily verify the code with different puppet versions.
Lorenz
Lorenz Schori
Well, after reading a big part of the puppet source, the rspec tests
and the wiki i'm still convinced that the puppet dependency handling
can be enhanced by implementing my original proposal. I now found a way
to provide this mechanism without altering the puppet core (at least
not patching it). Instead i've put together a function plugin which
injects the necessary code on demand. I've tested the code with puppet
versions 0.24.5 and lukes branch from github.
It is now possible to test the feature by getting the following two
files and play with them on a vanilla puppet installation. See the
first one on how to install and use the plugin:
http://znerol.ch/files/vertical_dependencies.rb
http://znerol.ch/files/vertical-dependencies-test.pp
Additional comments inline.
On Fri, 24 Oct 2008 11:48:17 -0500
Luke Kanies <lu...@madstop.com> wrote:
> [...]
> I actually tried this maybe 1.5 years ago, and it was basically
> completely untenable and pissed everyone off.
Hm. Could you please be more specific on that? I was not able to find
the code / comments in the git history.
> The crux is: What happens if you need to delete a resource in order
> to correctly configure another resource? That is, correct
> configuration of a given resource class requires negation of some
> resource.
>
> In that case, we would incorrectly set order, which would then break
> everything.
No, you just use "require" to depend on the other resource in exact
the same order like you did before. "above" and "below" metaparameters
do not alter the behavior of "require" and "before" in any way.
> The only real way to do something like this is to have class-level
> 'ensure'-equivalents: We'd need to be able to talk about the whole
> service class being present or running or whatever, and then the
> 'ensure' state of all contained resources would be bit-flipped
> depending on what we did with that class-level state.
I've commented on that before but i might revise my statement. I use
classes almost exclusively for grouping together defines and therefore
i'm not very concerned about class-level parameters. I somehow worked
around them. On the other hand i see that there are some use cases for
parametrized classes but i just would think about them as singleton
defines - which is what they actually are in reality.
Lorenz
jerico
> I've commented on that before but i might revise my statement. I use
> classes almost exclusively for grouping together defines and therefore
> i'm not very concerned about class-level parameters.
currently impose a very strict installation hierarchy "by convention"
anyway. So "above"/"below" comes naturally to me.
Therefore I like your idea of "above" and "below" as it solves some of
my problems as well although I unfortunately do not find the time to
fix the code as you did. Thanks for your contribution! I'll try it out
later.
My current workaround is: I am using a custom "installed" parameter
with my definitions to emulate the behaviour you are providing through
"above"/"below". Then I use a couple of "if" statements within my
definitions that ensure that enable, ensure, require, before, notify
and subscribe get fixed depending on installed=false or
installed=true. Like that I can uninstall resources as a bundle and in
the right order without causing damage. I leave them in "uninstalled"
state until all nodes have updated their state then I can remove the
definition instances from the configuration.
I prefer this approach over class inheritance (enable/disable):
1) Above all: I don't like classes at all for state management as they
do not accept parameters and force me to use global variables (which I
won't, see links below)
2) The installed-variable can be easily passed on to nested lower-
level definitions. If you use class inheritance however you usually
have to fix more than just one variable when you want to flip state
from installed to uninstalled or vice versa.
3) And of course I generally don't like the concept to "hard code"
state into class names as it's more difficult to maintain when you
extend or refactor your classes/definitions.
It's obvious that I'd prefer a "native" solution to the problem rather
than having to work around it with all the practical problems implied.
> I somehow worked
> around them. On the other hand i see that there are some use cases for
> parametrized classes but i just would think about them as singleton
> defines - which is what they actually are in reality.
between classes and defines. This was not such a problem if they were
really syntactically equivalent. But that's not the case. Classes
don't accept parameters and therefore invite use of global variables
which I believe is a very bad idea. Definitions are not inheritable
and do not provide namescoped variables (or better: constants).
See this post for a rather complete discussion of the conceptual
similarities/differences between classes and defines (and scope for
improvement) that I had with Luke:
http://groups.google.com/group/puppet-dev/browse_thread/thread/5e211ec60186c077?hl=en#
This discussion is based on the following earlier post
http://groups.google.com/group/puppet-dev/browse_thread/thread/23d5e56f1770ec85?hl=en#
and on my feature request #1645
@Luke: I am linking this so that you don't have to repeat your
arguments again.
Florian
Lorenz Schori
On Tue, 4 Nov 2008 04:04:01 -0800 (PST)
jerico <jeric...@gmail.com> wrote:
>
> Hi Lorenz,
>
> > I've commented on that before but i might revise my statement. I use
> > classes almost exclusively for grouping together defines and
> > therefore i'm not very concerned about class-level parameters.
>
> As you I am using classes nearly exclusively to bundle definitions. I
> currently impose a very strict installation hierarchy "by convention"
> anyway. So "above"/"below" comes naturally to me.
>
> Therefore I like your idea of "above" and "below" as it solves some of
> my problems as well although I unfortunately do not find the time to
> fix the code as you did. Thanks for your contribution! I'll try it out
> later.
Unfortunately the plugin-approach does only work if puppet runs in
standalone mode because only the server knows about function plugins.
perhaps its possible to trick puppetd into loading the code if it is
provided as a type/provider plugin instead of a function plugin.
It is still possible to patch puppet (>0.24.5) to get above/below
working in client/server mode using the following patch:
<http://znerol.ch/files/0001-above-and-below-vertical-dependency-metaparameters.patch>
My biggest problem now is that i have to eliminate virtually every
autorequire by explicitly specifying above/below because otherwise its
likely that the dependency tree gets messed up when resources are
deleted.
> My current workaround is: I am using a custom "installed" parameter
> with my definitions to emulate the behaviour you are providing through
> "above"/"below". Then I use a couple of "if" statements within my
> definitions that ensure that enable, ensure, require, before, notify
> and subscribe get fixed depending on installed=false or
> installed=true. Like that I can uninstall resources as a bundle and in
> the right order without causing damage. I leave them in "uninstalled"
> state until all nodes have updated their state then I can remove the
> definition instances from the configuration.
>
> I prefer this approach over class inheritance (enable/disable):
> [...]
> It's obvious that I'd prefer a "native" solution to the problem rather
> than having to work around it with all the practical problems implied.
For sure, me too.
> [...]
>
> See this post for a rather complete discussion of the conceptual
> similarities/differences between classes and defines (and scope for
> improvement) that I had with Luke:
>
> [...]
I'm following that thread with great interest.
Lorenz
Luke Kanies
> On Fri, 24 Oct 2008 11:48:17 -0500
> Luke Kanies <lu...@madstop.com> wrote:
>
> Ok, first i want to say that i'm very happy with puppet and the way it
> works. The require-thing is just one little annoyance which really
> bugged me long enough that i decided to look at it closer. So if i
> criticize some puppet internals i try to find the most appropriate
> fair
> and clear words. I don't intend to be rough against you luke and all
> the other developers who realized this wonderful system.
Heh, this is by no means rough, other than it being a lot of email to
reply to (guess it's been a while since I had 10k words a day in
threads).
>
>> [...]
>>
>> The crux is: What happens if you need to delete a resource in order
>> to correctly configure another resource? That is, correct
>> configuration of a given resource class requires negation of some
>> resource.
>>
>> In that case, we would incorrectly set order, which would then break
>> everything.
>
> I agree, but this is a special case. Let's treat that special and
> carefully instead of complicate the standart cases. I argue that it
> should be easy to add *and* remove resources which depend on each
> other
> without requiring users to work around dependency-problems for the
> ensure => absent case for each and every resource. Additionally with
> the "above"/"below" solution "require" and "before" will not be
> touched
> in any way, they continue to behave like before. This two dependency
> models may coexist as long as they will not be intermixed heavily -
> that's actually a thing which i want to test more in depth.
>
> I've attached POC patches and some test cases to illustrate the
> situation. Note the mybind-define. This is one of my most used
> patterns
> and the reason i'm not happy with the dependency system.
The docs attached to your code makes it more clear what you mean --
you're talking about providing relationship parameters that
automatically switch direction depending on deletion or creation.
This is much clearer to me than your previous, longer description, and
I tentatively like the idea. I'm not sure about the terms 'above' and
'below', partially because I'm not sure about the existing
relationship terms, but I like the idea.
I've got multiple open tickets on existing relationship problems,
including the long-standing goal of getting relationship specification
into the language rather than relying on attributes, so your request
is, for me, partially a question of how it interacts, if at all, with
other goals for relationships. It seems straightforward, though, and
your attachments demonstrate that it largely is (although I would do
it a bit differently).
I'll also mention: Look through transaction.rb for 'delete' -- I made
some inroads to thinking about this problem a while ago, as previously
mentioned, and it pays to understand previous approaches.
>
>> The only real way to do something like this is to have class-level
>> 'ensure'-equivalents: We'd need to be able to talk about the whole
>> service class being present or running or whatever, and then the
>> 'ensure' state of all contained resources would be bit-flipped
>> depending on what we did with that class-level state.
>
> I don't thing that this is necessary. I do often something like this:
>
> class apache {
> # general defines go here
> package { "apache2": ensure => installed }
> service { "apache2": enable => false, ensure => stopped }
> }
>
> class apache::enable inherits apache {
> # config stuff goes here
> Service["apache"] { enable => true, ensure => running }
> }
>
> class apache::disable inherits apache {
> # special clean up stuff (rarely used)
> Service["apache"] { enable = false, ensure => stopped }
> }
>
> Then i include apache or apache::enable in specialized classes or even
> inherit from apache::enable and include them in node definitions. I
> think it is better to not try to make puppet smarter than it can be.
> The
> bitflipping approach sounds to me like a very bad solution to a not so
> bad problem - at least if i did understand your short description
> correctly.
I, um, hate this. Whether a service is enabled or disabled seems to
me to be a fundamental aspect of providing that service -- a node is
saying that it understands apache but is not running it, vs. something
like it understands how to enable or disable apache. You've got basic
terminology problems that expose the issue here -- you've got a verb
as your class name (enable and disable). Even switching them to
adjectives (enabled and disabled) doesn't solve the problem, because
they should be nouns.
>
>
> Lorenz
>
> >
> <above-case.pp><require-case-broken.pp><require-case-fixed.pp>diff --
> git a/lib/puppet/type.rb b/lib/puppet/type.rb
> index 85f72f7..88b855a 100644
> --- a/lib/puppet/type.rb
> +++ b/lib/puppet/type.rb
> @@ -1533,9 +1533,18 @@ class Type
> self.fail "Could not retrieve dependency '%s' of
> %s" % [reference, @resource.ref]
> end
>
> + deleting = (@resource[:ensure] == :absent ||
> @resource[:ensure] == :purge)
> + if self.class.direction == :inoncreation
> + direction = deleting ? :out : :in
> + elsif self.class.diroction == :inondeletion
> + direction = deleting ? :in : :out
> + else
> + direction = self.class.direction
> + end
There's already a 'deleting?' method on the Type class, so each
resource can define its own idea of deletion; that method should be
used, rather than relying on testing it here.
>
> # Are we requiring them, or vice versa? See the
> method docs
> # for futher info on this.
> - if self.class.direction == :in
> + if direction == :in
> source = object
> target = @resource
> else
> @@ -1669,6 +1678,16 @@ class Type
> This will restart the sshd service if the sshd config
> file changes.}
> end
>
> + newmetaparam(:above, :parent =>
> RelationshipMetaparam, :attributes => {:direction
> => :inoncreation, :events => :NONE}) do
> + desc %{This parameter works like **require** on creation of
> the
> + ressource (ensure => present) and behaves like **before**
> on deletion.}
> + end
> +
> + newmetaparam(:below, :parent =>
> RelationshipMetaparam, :attributes => {:direction
> => :inondeletion, :events => :NONE}) do
> + desc %{This parameter works like **require** on deletion of
> the
> + ressource (ensure => absent) and behaves like **before** on
> creation.}
> + end
> +
[deleted duplicate code]
>
> ### test with plain types file and mount requiring file resources ###
> # $ensure = present
> # puppet require-case-broken.pp
> notice: //Node[default]/File[/tmp/puppet-testmnt]/ensure: created
> notice: //Node[default]/Mount[/tmp/puppet-testmnt]/ensure: defined
> 'ensure' as 'mounted'
> notice: //Node[default]/Mount[/tmp/puppet-testmnt]: Refreshing self
This is exactly the kind of problem that caused me to start thinking
about deletion, a good while ago, so I understand your frustration.
--
Kai's Example Dilemma:
A good analogy is like a diagonal frog.
Luke Kanies
>
> Hell Again,
>
> Well, after reading a big part of the puppet source, the rspec tests
> and the wiki i'm still convinced that the puppet dependency handling
> can be enhanced by implementing my original proposal. I now found a
> way
> to provide this mechanism without altering the puppet core (at least
> not patching it). Instead i've put together a function plugin which
> injects the necessary code on demand. I've tested the code with puppet
> versions 0.24.5 and lukes branch from github.
>
> It is now possible to test the feature by getting the following two
> files and play with them on a vanilla puppet installation. See the
> first one on how to install and use the plugin:
> http://znerol.ch/files/vertical_dependencies.rb
> http://znerol.ch/files/vertical-dependencies-test.pp
Great. Are these available in a git repo somewhere?
>
> Additional comments inline.
>
> On Fri, 24 Oct 2008 11:48:17 -0500
> Luke Kanies <lu...@madstop.com> wrote:
>
>> [...]
>> I actually tried this maybe 1.5 years ago, and it was basically
>> completely untenable and pissed everyone off.
>
> Hm. Could you please be more specific on that? I was not able to find
> the code / comments in the git history.
Basically, I didn't think to do what you're doing here -- add new
params whose direction flipped depending on whether you're deleting or
creating.
>
>> The crux is: What happens if you need to delete a resource in order
>> to correctly configure another resource? That is, correct
>> configuration of a given resource class requires negation of some
>> resource.
>>
>> In that case, we would incorrectly set order, which would then break
>> everything.
>
> No, you just use "require" to depend on the other resource in exact
> the same order like you did before. "above" and "below" metaparameters
> do not alter the behavior of "require" and "before" in any way.
Yeah, this was my confusion based on what you were trying.
>
>> The only real way to do something like this is to have class-level
>> 'ensure'-equivalents: We'd need to be able to talk about the whole
>> service class being present or running or whatever, and then the
>> 'ensure' state of all contained resources would be bit-flipped
>> depending on what we did with that class-level state.
>
> I've commented on that before but i might revise my statement. I use
> classes almost exclusively for grouping together defines and therefore
> i'm not very concerned about class-level parameters. I somehow worked
> around them. On the other hand i see that there are some use cases for
> parametrized classes but i just would think about them as singleton
> defines - which is what they actually are in reality.
Okay. As jerico mentions, this is being covered heavily in a
different thread, so I'll leave it there.
--
Brand's Shortcut:
The only way to predict the future is to make sure it stays
exactly the same as the present.
Lorenz Schori
On Tue, 4 Nov 2008 12:11:54 -0600
Luke Kanies <lu...@madstop.com> wrote:
> > files and play with them on a vanilla puppet installation. See the
> > first one on how to install and use the plugin:
> > http://znerol.ch/files/vertical_dependencies.rb
> > http://znerol.ch/files/vertical-dependencies-test.pp
>
> Great. Are these available in a git repo somewhere?
Fresh and shiny (hopefully):
git remote add znerol git://znerol.ch/puppet
Changes against master are in branch feature/master/vertdep. I did not
include the plugin-variant because it only works if puppet runs in
standalone mode like i mentioned in another posting.
Tests are lacking, i was not able to figure out how to test
metaparameters using rspec. Some suggestions on where to start? Which
tests to look at for reference?
I did some more research in redmine and found #428 (Deletion should
invert sort order) which seems to be very related to what i am after.
I'm thinking about some general model for dependency handling and will
post again in a new therad if i'm able to come up with something
usable...
Lorenz
Luke Kanies
It's a nice small patch; I'll have to help you get tests for it before
we consider applying it, but that shouldn't be too hard.
I've got a couple of issues with it, though. The 'Type' class already
has a 'deleting?' method you can use to determine whether a resource
is being deleted -- currently it has the same behaviour you're using,
but it allows a resource type to override what deletion means. This
is useful for things like package purging -- it's a form of deletion
that doesn't use :absent.
Also, what's with setting 'self.class.direction = (ens
== :absent ? :out : :in)'? That sets a class instance variable, which
means it will affect every above/below parameter, which can't be what
you mean. I think the only way to get the behaviour you want is to do
some refactoring -- maybe use an instance-level 'direction' method
that normally just returns the class's direction and then override the
direction instance method in theseparams (preferable), or refactor the
'direction' method to accept the ensure value as an argument.
--
The conception of two people living together for twenty-five years
without having a cross word suggests a lack of spirit only to be
admired in sheep. --Alan Patrick Herbert
Lorenz Schori
On Sun, 9 Nov 2008 23:16:03 -0600
Luke Kanies <lu...@madstop.com> wrote:
> [...]
> > I did some more research in redmine and found #428 (Deletion should
> > invert sort order) which seems to be very related to what i am
> > after. I'm thinking about some general model for dependency
> > handling and will post again in a new therad if i'm able to come up
> > with something usable...
>
> It's a nice small patch; I'll have to help you get tests for it
> before we consider applying it, but that shouldn't be too hard.
>
> I've got a couple of issues with it, though. The 'Type' class
> already has a 'deleting?' method you can use to determine whether a
> resource is being deleted -- currently it has the same behaviour
> you're using, but it allows a resource type to override what deletion
> means. This is useful for things like package purging -- it's a form
> of deletion that doesn't use :absent.
I know that and i'm not using this mechanism by purpose. The problem is
that there are resources like exec which are not deletable at all. At
first I simply ignored that fact and promptly ran into problems with
exec inheriting a "below"-metaparam from an enclosing define and
producing dependency cycles. Execs are not deletable and therefore we
must respect that fact and throw errors if an attempt is made to assign
above/below metaparams. The easiest way to do this is just to check for
the ensure attribute. If it's not there we have a problem. If some type
for some reason behaves completely different from the built-in ones, it
is sort of likely that the above/below metaparams will not work
properly for that one too. Because of this i think that it is safest to
stick with the ensure attribute.
> Also, what's with setting 'self.class.direction = (ens
> == :absent ? :out : :in)'? That sets a class instance variable,
> which means it will affect every above/below parameter, which can't
> be what you mean. I think the only way to get the behaviour you
> want
I am fully aware of the fact that direction is a class instance
variable but puppet does not multithread and thus it was the way of
least resistance... Not the cleanest one though.
> is to do some refactoring -- maybe use an instance-level 'direction'
> method that normally just returns the class's direction and then
> override the direction instance method in theseparams (preferable),
> or refactor the 'direction' method to accept the ensure value as an
> argument.
OK, tried this in two new branches:
refactor/master/type-direction
feature/refactor-type-direction/vertdep
Lorenz
Sven Mueller
use
"before" and "after" (for now an alias to "require") if you simply want
the old behaviour (run before or after another resource, wether it is
deleted or added).
use
"need" (above in Lorenzs terminology) and "isneeded" (below) if you want
the new behaviour (actually needs other resource to be there to work, or
needs to be there for other resource to be useful)
(I was considering "require_available" for "need" and "available_before"
for "isneeded", but found them a bit misleading and -frankly- too long.)
regards,
Sven
Luke Kanies
That's a good point.
I'd still prefer to have the functionality pushed into a method.
Maybe have a 'deletable?' method to go along with the 'deleting?'
method.
>
>> Also, what's with setting 'self.class.direction = (ens
>> == :absent ? :out : :in)'? That sets a class instance variable,
>> which means it will affect every above/below parameter, which can't
>> be what you mean. I think the only way to get the behaviour you
>> want
>
> I am fully aware of the fact that direction is a class instance
> variable but puppet does not multithread and thus it was the way of
> least resistance... Not the cleanest one though.
It's not so much about multithreading as it is code design. It's a
relatively easy design problem to fix.
You're right that I don't see a place where it would cause a problem,
but it might as well be clean.
>
>> is to do some refactoring -- maybe use an instance-level 'direction'
>> method that normally just returns the class's direction and then
>> override the direction instance method in theseparams (preferable),
>> or refactor the 'direction' method to accept the ensure value as an
>> argument.
>
> OK, tried this in two new branches:
> refactor/master/type-direction
> feature/refactor-type-direction/vertdep
The code looks good, barring the lack of tests. Now we just need to
agree on terminology.
I don't think above/need or below/isneeded do a good job of capturing
the functionality of these parameters.
Anyone else have any other proposals? Can we restate the purpose to
make it more clear? Something like:
- Declares that the related resource must be created first and deleted
after
- Delcares that the related resource must be created after and deleted
before
It's almost like a container relationship -- the container must be
created before its contained resources, and the contained resources
must be deleted before their container. Maybe 'contains' and
'containedby'?
--
Today at work an ethernet switch decided to take the 'N' out of NVRAM
-- Richard Letts
Paul Lathrop
I vote for 'contains' and 'container', myself.
--Paul
Luke Kanies
>> It's almost like a container relationship -- the container must be
>> created before its contained resources, and the contained resources
>> must be deleted before their container. Maybe 'contains' and
>> 'containedby'?
>
> I vote for 'contains' and 'container', myself.
Seem like good choices.
--
When one admits that nothing is certain one must, I think, also admit
that some things are much more nearly certain than others.
-- Bertrand Russell
Lorenz Schori
Luke Kanies <lu...@madstop.com> wrote:
>
> On Dec 1, 2008, at 1:33 PM, Paul Lathrop wrote:
>
> >> It's almost like a container relationship -- the container must be
> >> created before its contained resources, and the contained resources
> >> must be deleted before their container. Maybe 'contains' and
> >> 'containedby'?
> >
> > I vote for 'contains' and 'container', myself.
>
>
> Seem like good choices.
Well, i probably played too much with my lego-bricks when i was a
child, so above and below sounds very natural to me. Did you guys had
babooshka puppets?
Lorenz
Luke Kanies
I think I know what you mean, but it just seems like a more natural
relationship.
I see now what you mean about above/below being reasonable terms; you
have to add the bottom lego then the above lego, and vice versa for
deletion. I have to say, though, legos weren't exactly my first
thought for the terms. :)
Is there another kind of relationship that naturally implies this LIFO
sorting?
--
I have an answering machine in my car. It says, "I'm home now. But
leave a message and I'll call when I'm out. -- Stephen Wright
Paul Lathrop
> I see now what you mean about above/below being reasonable terms; you
> have to add the bottom lego then the above lego, and vice versa for
> deletion. I have to say, though, legos weren't exactly my first
> thought for the terms. :)
>
> Is there another kind of relationship that naturally implies this LIFO
> sorting?
Just brainstorming, not advocating any of these:
supports/needs
provides/consumes
carries/rides
precedes/succeeds
roots/branches
foundation/structure
Okay, feel like I'm reaching now.
--Paul
Luke Kanies
What resource pairs can we think of that might have this kind of
relationship?
Directory/file
Package/service
Group/user
Maybe file/service?
I can't think of others.
All but the last could reasonably said to be containment relationships.
--
Charm is a way of getting the answer yes without asking a clear
question. -- Albert Camus
Brice Figureau
> On Dec 2, 2008, at 2:54 PM, Lorenz Schori wrote:
>
>> On Mon, 1 Dec 2008 13:45:02 -0600
>> Luke Kanies <lu...@madstop.com> wrote:
>>
>>> On Dec 1, 2008, at 1:33 PM, Paul Lathrop wrote:
>>>
>>>>> It's almost like a container relationship -- the container must be
>>>>> created before its contained resources, and the contained resources
>>>>> must be deleted before their container. Maybe 'contains' and
>>>>> 'containedby'?
>>>> I vote for 'contains' and 'container', myself.
>>>
>>> Seem like good choices.
>> Well, i probably played too much with my lego-bricks when i was a
>> child, so above and below sounds very natural to me. Did you guys had
>> babooshka puppets?
>
>
> I think I know what you mean, but it just seems like a more natural
> relationship.
>
> I see now what you mean about above/below being reasonable terms; you
> have to add the bottom lego then the above lego, and vice versa for
> deletion. I have to say, though, legos weren't exactly my first
> thought for the terms. :)
>
> Is there another kind of relationship that naturally implies this LIFO
> sorting?
Parents/childrens ?
Marcin Owsiany
Parents _usually_ die _before_ children..
--
Marcin Owsiany <mar...@owsiany.pl> http://marcin.owsiany.pl/
GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216
"Every program in development at MIT expands until it can read mail."
-- Unknown
Lorenz Schori
Me neither. Container relationships seem to work well. Only i'm not
able to come up with reasonable terms expressing these relations
well... inside/outside (too general but easy words) perhaps enclose,
hold, embed but i have difficulties to figure out suited antonyms.
Lorenz
Luke Kanies
We can start with 'contains' and 'containedby', unless someone else
has a better term than 'containedby'.
--
There are no such things as applied sciences, only applications of
science. -- Louis Pasteur
Paul Lathrop
> We can start with 'contains' and 'containedby', unless someone else
> has a better term than 'containedby'.
I stand firmly by my suggestion of 'container' instead of 'containedby'
--Paul
Luke Kanies
Ah, right. That works.
--
The world tolerates conceit from those who are successful, but not
from anybody else. -- John Blake
Andrew Shafer
I'm a little late to the party, but I have some thoughts. I actually don't mind 'above' and 'below', but then I didn't think membership => inclusive was intuitive... :/ (I don't really care what the words are as long is there some sensible semantics when it is all said and done.)
Looking at the patches and puppet code, I'm wondering if anyone has tried setting these up when any of the resources are using other metaparameters? Luke please correct me where I'm wrong, but I think you have to do more than just reverse the direction of the edges if you have a resource with both an above/below relationship and a before/after with another resource.
Maybe this will never come up, or you make them mutually exclusive.
To show my concern with pictures:
If you have
A contains B
A after X
The original graph looks like: Top -> X -> A ->B
and then when you flip it: Top -> X -> A <- B
I believe every resource has an implicit relationship with Top so now you have:
Top -> X-> A
|__>B___^
If I understand correctly hilarity ensues, people blame Puppet and say it is possessed and indeterminate.
And what happens if it looks like this:
A contains B and D
B contains C
Top -> X -> A -> B -> C
|__>D
I might be imagining things, but I think you would have to find all the terminal nodes of the sub-graph under A and attach them to X. (or build up all those implicit edges at the outset.)
Has anyone tried this and tested the behavior?
Lorenz Schori
On Fri, 5 Dec 2008 00:52:29 -0700
"Andrew Shafer" <and...@reductivelabs.com> wrote:
> Has anyone tried this and tested the behavior?
Yes i did some tests and it is like you say. If above/below is combined
with require/before we are able to create ugly dependency cycles
depending on the direction of some ressources (ensure => present vs.
ensure => absent). However puppet will not try to be smart about those
things, it will just fail and report a dependency cycle. Especially in
combination with autorequire things may get really more complicated
than with current require/before.
The reason that i did publish this patches is that i wanted to express
the functionality i'm after as (working) code, so people are able to
test, understand, give feedback and do more research on the matter.
Lorenz
Luke Kanies
What did you do to exhibit the dependency cycles? I'm having a hard
time envisioning a normal use case resulting in them.
--
The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw
Luke Kanies
Not quite -- everyone has an edge to Top, but before the graph gets
sorted, Top gets removed, and only relationship edges are retained.
So, if you made an explicit dependency on a class, then you could do
this, but it wouldn't happen all the time or anything.
I'm of the opinion that we shouldn't need to make the different
relationship types mutually exclusive -- with great power comes great
responsibility, and if you're foolish enough to make relationships
that conflict, then you deserve the dependency cycle.
The only "strange" thing, I think, in this situation is that you'd
potentially have different cycles when deleting than when not (or,
more likely, new cycles where none existed).
Either way, I don't think the above change matters too much, as you're
effectively saying X has a relationship to the A/B subgraph, and that
subgraph *should* be able to be moved around as you want.
>
> And what happens if it looks like this:
> A contains B and D
> B contains C
>
> Top -> X -> A -> B -> C
> |__>D
>
> I might be imagining things, but I think you would have to find all
> the terminal nodes of the sub-graph under A and attach them to X.
> (or build up all those implicit edges at the outset.)
>
> Has anyone tried this and tested the behavior?
I hope to do so today.
--
It is a mistake to think you can solve any major problems just with
potatoes. --Douglas Adams