Cross-module dependencies
jcbollinger
couple of days, or else was unable to do so,
On Jan 24, 3:28 am, Felix Frank <felix.fr...@alumni.tu-berlin.de>
wrote:
>
> there was a discussion in the "can we deprecate defined() in Telly"
> thread about how we can even begin to design Forge modules without it.
>
> A recurring problem is that multiple modules rely on certain packages,
> and there is no good model (yet) to unite their resource declarations.
> Therefore it's a common (although imho disgusting) workaround to do
> things like
> if !defined(Package[foo]) { package { "foo": ensure => installed } }
>
> On 01/20/2012 11:34 PM, Cody wrote:
>
> > Defining all somewhat common packages in a central location becomes
> > unrealistic when you no longer "control" the code that is in every
> > module you use. If you obtain five modules from the forge and they
> > all require a specific package and so all define that package your not
> > going to convince, nor is it a good design to require everyone to move
> > the package definitions from that collection of modules. They need to
> > function as a collection out of the box.
>
> Agreed. How can this be accomplished?
I'm not sure it can be, though I have some ideas for how we can do
better than we do now.
To start, consider the extreme case where modules have conflicting
essential requirements. For example, they require packages that
declare conflicts with each other, or they differ about whether some
service must be enabled or disabled. Such modules cannot *ever* work
correctly together, out of the box or otherwise, therefore it is
impossible to achieve a system that ensures that any random collection
of modules will automatically work together.
Furthermore, I think that its probably the wrong goal that even
compatible modules should always work together automatically. The
goal should be that compatible modules work together without
modification, but it is altogether reasonable for that to require
extra code somewhere else (e.g. extra classes assigned to the node,
intermodule relationships defined, etc.).
> Perhaps there needs to be some kind of "Forge common" module that by
> policy can only ever declare virtual resources (packages are a prominent
> example).
> A user who wishes to retain the capability of using modules from the
> Forge would be required to install this common module, and replace their
> own resource declarations with realizations of the common resources.
> For this to work, it's definitely a plus that you can override
> attributes in collections:
> Package<| title == "apache2": |> { ensure => "2.2.12" }
> ...although that does bear some caveats. Does this still work in recent
> versions?
>
> If we can take this for granted, all Forge modules can adhere to that
> same standard.
>
> This is a rough sketch of how things might possibly work, and surely has
> lots of wrinkles of its own. Still, I'm quite sure we need a proper way
> to rid ourselves of the horror that is the parse order dependent check
> for defined resources ;-)
If we must rely only on features already present in Puppet, then I
think that's heading in the right direction. I doubt it's feasible to
rely on a single "Forge Common" module, however. Aside from the
problem of maintaining a "Common" module as other modules are created
and maintained, there is also the same essential problem I began with:
different modules may have conflicting requirements.
With respect to a given module, we need to distinguish between two
three types of resources:
1) Resources owned by that module
2) All other resources
Modules provide definitions of resources that they own. For the most
part, those definitions should be virtual to avoid unnecessary inter-
module coupling, but some resources are reasonable to define
concretely. Modules may realize virtual resources belonging to other
modules (without necessarily needing to know which module actually
provides the definition), but they must not override properties of
resources they do not own.
The set of resources owned by a module and the set of other resources
it depends on are parts of its external interface, and modules whose
sets of owned resources overlap are inherently incompatible. Because
of that source of incompatibility, modules should seek to keep their
lists of owned resources small. But how, then, can optional inter-
module dependencies be handled, or equivalently, how can a module be
made able both to interoperate and to stand on its own? I see two
elements to this:
1) It is the site's responsibility to ensure that all "other"
resources required by each module in use be provided. That is the
role that Felix's "Forge Common" module is aimed at, though I don't
much care for that particular solution. Instead, I think in many
cases it will be necessary for sites to provide at least some resource
definitions via local custom modules.
2) Modules can ease the task for users by providing *optional* classes
virtually defining some or all of the "other" resources they need.
Where there are no conflicts to deal with, users could assign those
classes to nodes along with the module's main classes, and thereby
avoid any need to define the needed resources themselves.
John
jcbollinger
On Jan 26, 11:00 am, jcbollinger <John.Bollin...@stJude.org> wrote:
> If we must rely only on features already present in Puppet, then I
> think that's heading in the right direction.
area, then there may be altogether different approaches available. In
particular, it is useful to recognize that dependencies are not just
on a particular resource generally -- rather, they are on a particular
resource having certain specific properties. For example, a web
server module doesn't just rely on, say, Package['httpd']. Rather, it
relies on that package having an 'ensure' parameter different from
'absent' and 'purged'. Puppet DSL does not currently have a means to
express that.
Consider, then, a new metaresource type, Constraint. The purpose of
the Constraint resource type would be to allow multiple unrelated
classes to collaborate on defining the properties of a single
resource, and it would do so by allowing classes to limit the values
that chosen resource properties may have.
At compilation time, Puppet would collect and combine all the
constraints on any particular resource, and use the results to set
unspecified property values and validate specified ones. Usage might
look something like this:
constraint { 'webserver-httpd_package-present':
resource => Package['httpd'],
property => 'ensure',
forbidden_value => [ 'absent', 'purged' ],
# also available: allowed_value
# maybe: default_value
}
Not only would this nicely meet the needs of different modules to
express their requirements on shared resources, it would also make it
much easier to recognize resource conflicts. If Puppet automatically
generated empty resource definitions to constrain when it discovered
constraints on otherwise-undefined resources, then that would also
overcome the problem of deciding where to define particular resources.
I can imagine many -- perhaps most -- resource definitions being
replaced or supplemented by constraint declarations.
John
Felix Frank
thanks for coming up with such elaborate ideas, your input to this group
adds a lot of meat to many discussions.
I can agree with a lot of what you wrote, barring the following remarks:
On 01/26/2012 06:00 PM, jcbollinger wrote:
> Modules provide definitions of resources that they own. For the most
> part, those definitions should be virtual to avoid unnecessary inter-
> module coupling, but some resources are reasonable to define
> concretely.
Jeff has made a strong point against using virtual resources in modules
at all, causing me to shift my own views as well.
If I understand him correctly, one of the chief problems is the high
probability of accidental collection/realisation of such resources by
the end user's manifest.
On 01/26/2012 06:48 PM, jcbollinger wrote:
> I can imagine many -- perhaps most -- resource definitions being
> replaced or supplemented by constraint declarations.
The model is intriguing, but gives me another usability headache.
Wouldn't this put an end to self-contained modules?
I wrote in a latter mail (is this the same thread? Sorry, I use this
only through Thunderbird and get confused sometimes) how I see need for
explicit module dependencies and a system that can automatically
download required modules from the forge. I can see this supplementing
your idea of constraints nicely, but without it, downloading modules
could quickly become a nightmare for users.
Cheers,
Felix
Walter Heck
On Fri, Jan 27, 2012 at 15:20, Felix Frank
<felix...@alumni.tu-berlin.de> wrote:
> how I see need for
> explicit module dependencies and a system that can automatically
> download required modules from the forge. I can see this supplementing
> your idea of constraints nicely, but without it, downloading modules
> could quickly become a nightmare for users.
There's something else we need to think about here. Some modules have
a soft/conditional requirement for other modules. What I mean is that
if you don't use certain parts of a module, you don't need the module
that that part of the code refers to. the only decent way I can come
up with to solve that is to use what for instance in C is done with
#IFDEF. That way the module could just ignore modules that it doesn't
_really_ require.
I for instance have modules that allow you to use different backends
for monitoring or backups. If requirements were done automatically
based on the whole module, it would need a myriad of other modules,
only one of which is ever used.
cheers,
--
Walter Heck
--
follow @walterheck on twitter to see what I'm up to!
--
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily
Nick
> In particular, it is useful to recognize that dependencies are not just on a
> particular resource generally -- rather, they are on a particular resource
> having certain specific properties.
Yes.
Also: currently in Puppet one cannot say anything about a resource without a
declaration that you will "manage" it. (Unless perhaps that state happens to be
encapsulated by a Fact, which typically isn't and in many cases couldn't
feasibly be - like the case of file attributes.)
Therefore many "dependencies" are created only because of a need to check some
state of a resource - which one may not want or need to manage.
> Consider, then, a new metaresource type, Constraint. The purpose of the
> Constraint resource type would be to allow multiple unrelated classes to
> collaborate on defining the properties of a single resource, and it would do
> so by allowing classes to limit the values that chosen resource properties
> may have.
>
> At compilation time, Puppet would collect and combine all the constraints on
> any particular resource, and use the results to set unspecified property
> values and validate specified ones. Usage might look something like this:
>
> constraint { 'webserver-httpd_package-present':
> resource => Package['httpd'],
> property => 'ensure',
> forbidden_value => [ 'absent', 'purged' ],
> # also available: allowed_value
> # maybe: default_value
> }
>
> Not only would this nicely meet the needs of different modules to express
> their requirements on shared resources, it would also make it much easier to
> recognize resource conflicts. If Puppet automatically generated empty
> resource definitions to constrain when it discovered constraints on
> otherwise-undefined resources, then that would also overcome the problem of
> deciding where to define particular resources.
>
> I can imagine many -- perhaps most -- resource definitions being replaced or
> supplemented by constraint declarations.
Here's a slightly different angle.
(Note, I'll use capitalisation distinguish between the "resources" that exist
outside Puppet, and the "Resource" instances inside, which model them.)
I think there *is* a case to be made that Puppet needs a new "kind" of Resource
declaration. One which promises never to change the state of anything.
Immutability is, I think, the key to allowing multiple declarations of this sort
co-exist. Resources currently have to uniquely "own" a resource so that they can
safely change it. As I said, one doesn't always need or want that ownership: we
know the kind of baggage it carries.
Ideally we'd be able to separate out the aspects of a Resource which merely
assert what *should* be the case (ensure => present etc.) from those bits which
would then change the state of the resource if it deviates.
For the sake of discussion I'll call that former kind of declaration an "Assertion".
To briefly address the points from earlier email: Nan asked how would one
address merging, with respect to the following aspects of a Resource?
a) unifying before/requires attributes
b) unifying if/then/else constructs
c) auditing changes back to their source
d) unifying hash/array attributes
Well, when using Resources, yes, these make them very hard to unify in general.
However - although this needs more thought - if we could invent some way to
declare mere Assertions, they might do instead of Resources for many cases, and
it might be possible to unify them more simply, because:
a) Problems related to ordering mostly disappear,
...since nothing is being changed. (Of course, external things which might
change things could be a problem.)
b) If/then/else clauses can still make sense within definitions of custom
Assertions,
...if they can compose only other Assertions, and the conditions don't change.
c) Auditing is not a problem: since nothing is changed, there is nothing to
audit.
d) Hash/array attribute values would need to be resolved on a case-by-case
basis,
...depending on the semantics. When hash and arrays are semantically
representing are sets, this should be a straightforward "and" or "or" operation.
When the order matters, like a search path, or there is no obvious way to unify
two attributes, then this is an unresolvable contradiction and should generate
an error. There may be cases in between.
Possibly this doesn't fit all the use-cases which run into cross-module
dependency problems, but might significantly reduce the need to create the
dependencies in the first place.
Anyway, I need to get back to work, I'll try to say more in a later email.
Cheers,
N
Felix Frank
On 01/27/2012 02:52 PM, Walter Heck wrote:
> There's something else we need to think about here. Some modules have
> a soft/conditional requirement for other modules. What I mean is that
> if you don't use certain parts of a module, you don't need the module
> that that part of the code refers to. the only decent way I can come
> up with to solve that is to use what for instance in C is done with
> #IFDEF. That way the module could just ignore modules that it doesn't
> _really_ require.
thanks for pointing this out, but it has been covered (I think) in
another thread already:
On 01/19/2012 09:17 PM, Nick Fagerlund wrote:
> So, you can conditionally declare the rule if the defined type is
> available to the autoloader, and otherwise you don't attempt to manage
> the firewall and expect that the user has read the documentation and
> will make a hole for the service themselves.
>
> if defined(firewall::iptables::rule) {
> firewall::iptables::rule {'mysql_server':
> ...etc. etc.
> }
> }
>
> See? It's just a way to peek around at what the user has installed.
Thanks again to Nick for this quote, it keeps proving useful ;-)
Cheers,
Felix
jcbollinger
On Jan 27, 7:20 am, Felix Frank <felix.fr...@alumni.tu-berlin.de>
wrote:
>
> thanks for coming up with such elaborate ideas, your input to this group
> adds a lot of meat to many discussions.
>
> I can agree with a lot of what you wrote, barring the following remarks:
>
> On 01/26/2012 06:00 PM, jcbollinger wrote:
>
> > Modules provide definitions of resources that they own. For the most
> > part, those definitions should be virtual to avoid unnecessary inter-
> > module coupling, but some resources are reasonable to define
> > concretely.
>
> Jeff has made a strong point against using virtual resources in modules
> at all, causing me to shift my own views as well.
> If I understand him correctly, one of the chief problems is the high
> probability of accidental collection/realisation of such resources by
> the end user's manifest.
same resources, but each resource can be defined only once.
If modules' resource requirements were invariant then I could be
satisfied with documenting them and requiring the user to provide them
concretely; indeed, that's not very far from what I just proposed. On
the contrary, however, whether a particular resource is needed by some
module may depend on which of that module's classes are used, and on
which parameters and external data are supplied. This approach
therefore has its own inherent problem for users: to avoid declaring
unneeded resources.
Given a choice between the problems of avoiding declaring unneeded
resources and of avoiding collecting unwanted resources, I would
choose the latter. It is easier to deal with, and failure to
completely solve it is no more costly than failure to solve the
alternative problem.
> On 01/26/2012 06:48 PM, jcbollinger wrote:
>
> > I can imagine many -- perhaps most -- resource definitions being
> > replaced or supplemented by constraint declarations.
>
> The model is intriguing, but gives me another usability headache.
than anything else on the table:
(+) It enables modules to declare their requirements more precisely,
reducing the risk of modules compiling and applying correctly, but
nevertheless failing to achieve the desired configuration.
Constraints are the only proposal currently on the table that provide
that.
(+) A key aspect is that it allows Puppet to pinpoint and diagnose
conflicts at compilation time.
(+) Furthermore, it allows each module to express *only* its
requirements, ignoring properties that it doesn't care about (though
other modules might care about them). In fact, it rewards modules for
doing so by making them more interoperable.
(+) It does not rely on virtual resources (though it is compatible
with them), so it does not impose any risk of accidental resource
collection.
(+) It could be implemented in a manner that forestalls inter-module
parse order dependencies.
The main cost I see is:
(-) It creates a new distinction between resource properties that are
explicitly specified in resource definitions and those that are not.
For this reason (at least), such a feature could not be introduced
until Telly at the earliest.
That cost is incurred, however, only in manifest sets that use
Constraints, and then only with respect to resources that are targeted
by Constraints. It is anyway not a usability issue, but rather an
upgrade / conversion / compatibility issue.
> Wouldn't this put an end to self-contained modules?
*more* self-contained than they currently are.
> I wrote in a latter mail (is this the same thread? Sorry, I use this
> only through Thunderbird and get confused sometimes) how I see need for
> explicit module dependencies and a system that can automatically
> download required modules from the forge. I can see this supplementing
> your idea of constraints nicely, but without it, downloading modules
> could quickly become a nightmare for users.
system for Puppet modules. Although such a thing could undoubtedly be
implemented, I don't think it would be very suitable for addressing
the inter-module dependency problem. Package management systems such
as yum and apt rely on the fact that software dependency graphs are
more or less trees, but I don't think module dependency graphs tend to
be anywhere near as treelike. I suppose module repository managers
could enforce tree-shaped dependency graphs, but that imposes the
tool's requirements on the problem space instead of relying on a tool
that suits well in the first place.
I do not mean to say that a tool that automatically downloaded and
installed modules from the Forge would be useless -- far from it. I
just don't think that it would adequately address the inter-module
dependency issue, and therefore I would recommend that such a tool not
even try to do so. It solves an altogether different problem.
John
Felix Frank
On 01/27/2012 04:22 PM, jcbollinger wrote:
> From a usability perspective, I think this is a far better proposal
> than anything else on the table:
I've thought of another plus. Even though the design proposal adds to
the DSL (and complexity is generally to be avoided), it does so in a
manner that will not make it necessary for novices (or any end user) to
deal with the particular featureset, but instead limits its target
audience to developers of public modules.
>> Wouldn't this put an end to self-contained modules?
>
> No. What makes you think it would? I think it allows modules to be
> *more* self-contained than they currently are.
I was thinking along the lines of "currently modules will just install
the packages they need", for instance. But you're right of course -
currently modules can break each other because of this, so a way for the
compiler to clearly pinpoint reasons and locations of mismatches would
indeed be superior.
> I do not mean to say that a tool that automatically downloaded and
> installed modules from the Forge would be useless -- far from it. I
> just don't think that it would adequately address the inter-module
> dependency issue, and therefore I would recommend that such a tool not
> even try to do so. It solves an altogether different problem.
Again, I'm inclined to concur. The issue at hand is to stop modules from
breaking each other horribly. Once that's off the table, a module
management system is a whole new issue in and of itself.
Cheers,
Felix
jcbollinger
On Jan 27, 8:01 am, Felix Frank <felix.fr...@alumni.tu-berlin.de>
wrote:
>
> On 01/27/2012 02:52 PM, Walter Heck wrote:
>
> > There's something else we need to think about here. Some modules have
> > a soft/conditional requirement for other modules. What I mean is that
> > if you don't use certain parts of a module, you don't need the module
> > that that part of the code refers to. the only decent way I can come
> > up with to solve that is to use what for instance in C is done with
> > #IFDEF. That way the module could just ignore modules that it doesn't
> > _really_ require.
>
> thanks for pointing this out, but it has been covered (I think) in
> another thread already:
>
> On 01/19/2012 09:17 PM, Nick Fagerlund wrote:
And Nick's approach seems good and useful, but I don't think it
addresses Walter's concern. Walter observes that modules' runtime
inter-module dependencies can depend on how they are used, and he
argues, I think, that that complicates the problem of managing such
dependencies via a package management system.
For example, if module X depends on module Y only for feature
X::ySupport, and I intend never to use X::ySupport, then I might like
to avoid installing module Y. If I furthermore want to use module Y2
that is incompatible with module Y, then I might *need* to avoid
installing Y. There are ways to handle problems such as those, but it
all creates a lot of overhead to make everything work together
properly in such a scheme.
John
A hollow voice says "Plugh".
jcbollinger
On Jan 27, 7:58 am, Nick <oinksoc...@letterboxes.org> wrote:
> On 26/01/12 17:48, jcbollinger wrote:
>
> > In particular, it is useful to recognize that dependencies are not just on a
> > particular resource generally -- rather, they are on a particular resource
> > having certain specific properties.
>
> Yes.
>
> Also: currently in Puppet one cannot say anything about a resource without a
> declaration that you will "manage" it. (Unless perhaps that state happens to be
> encapsulated by a Fact, which typically isn't and in many cases couldn't
> feasibly be - like the case of file attributes.)
easily create a custom fact expressing any or all of the attributes of
a specific file. I think one could even do it for all of the files in
a directory, or even for an entire directory tree. Before too long,
however, that's going to become quite slow, and it would be difficult
to use on the Puppet side. I can't think of a persistent node state
detail that couldn't be expressed via a custom fact, but if you need
many such details, or if you can't be sure in advance which you will
need, then you do have a problem.
> Therefore many "dependencies" are created only because of a need to check some
> state of a resource - which one may not want or need to manage.
am not at all comfortable with that idea, though perhaps I could
become more so. It entails a much more fundamental change to Puppet
than anything else we have been discussing. Or did you have something
different in mind?
> Here's a slightly different angle.
> Ideally we'd be able to separate out the aspects of a Resource which merely
> assert what *should* be the case (ensure => present etc.) from those bits which
> would then change the state of the resource if it deviates.
>
> For the sake of discussion I'll call that former kind of declaration an "Assertion".
> Possibly this doesn't fit all the use-cases which run into cross-module
> dependency problems, but might significantly reduce the need to create the
> dependencies in the first place.
only Constraint, but I don't think there is yet an infrastructure to
support that part. However, I don't see at all how the Assertion idea
fits into the cross-module dependency picture. Perhaps this is
because I don't see the purpose of declaring resources that you do not
intend to manage.
When I consider cross-module dependencies, I think the usual case is
that a module *does* want ensure that particular resource properties
are managed, but it may not care about other properties, and it cares
about who owns Resources only insofar as someone has to own them and
it affects who can manage them. I can imagine wanting to _assert_
Resource properties only as a workaround for not being able actually
to manage them. Are there other reasons to want such a thing?
> Anyway, I need to get back to work, I'll try to say more in a later email.
is, I suspect I don't quite understand what you are hoping to
accomplish with it.
John
Trevor Vaughan
Hash: SHA1
I've been mulling this over and wanted to get opinions on an, ugly, but completely functional approach.
We've been talking about people supplying metadata to describe inter-class dependencies (inter-module dependencies really, but hear me out). With the
advent of the parameterized class, you could simply write all ifdef dependencies directly into a parameter like follows:
# Get this for the 'contains' function
include 'stdlib'
class foo (
# No requirements by default
$reqs = ['']
) {
if contains($reqs,'bar') {
include 'bar'
}
...some stuff...
if contains($reqs, 'bar') {
bar::baz { 'something': ... }
}
}
It's not elegant by any means, but it is functional and since (in theory) puppet only includes a class once, then all of the various includes would be
completely skipped.
If would be nice if, in this example, $reqs was actually a class metaparameter and Puppet would automatically try to include the class when passed
into that variable.
Benefits:
* Works with the current language structure
* Does what you want it to in terms of not needing defined
Drawbacks:
* Requires the user to have an explicit working knowledge of all modules and namespaces
* Adds a lot of random logic to the code (unless it becomes a metaparam of some sort)
Thanks,
Trevor
On 01/27/2012 08:52 AM, Walter Heck wrote:
> Hello,
>
> On Fri, Jan 27, 2012 at 15:20, Felix Frank
> <felix...@alumni.tu-berlin.de> wrote:
>> how I see need for
>> explicit module dependencies and a system that can automatically
>> download required modules from the forge. I can see this supplementing
>> your idea of constraints nicely, but without it, downloading modules
>> could quickly become a nightmare for users.
> There's something else we need to think about here. Some modules have
> a soft/conditional requirement for other modules. What I mean is that
> if you don't use certain parts of a module, you don't need the module
> that that part of the code refers to. the only decent way I can come
> up with to solve that is to use what for instance in C is done with
> #IFDEF. That way the module could just ignore modules that it doesn't
> _really_ require.
>
> I for instance have modules that allow you to use different backends
> for monitoring or backups. If requirements were done automatically
> based on the whole module, it would need a myriad of other modules,
> only one of which is ever used.
>
> cheers,
- --
Trevor Vaughan
Vice President, Onyx Point, Inc.
email: tvau...@onyxpoint.com
phone: 410-541-ONYX (6699)
pgp: 0x6C701E94
- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEcBAEBAgAGBQJPJBXGAAoJECNCGV1OLcypcXkH/3Y2nqqGgJzAKg9YVj/DjiB7
8zbtA7/nVvC8LwtIwwGi7jY+VcbietGwNC8JOoxnTdFN4dCb1xsAcTqzt8p/NXHE
HhwGIG9YGaMoZzvwtfUGc6wrOeqxLvInq2g6e0Qk5QkhBZVg7T5DV4/mvXfheZOR
n1mENjPNMoRONifb24PqxK91CbRtBmJGxEX8b6pDB529oU6aZxNQi6xSn1KSkCJM
SZjVaDoxPqHC4V9L3/J34Rq8H96tfMvTHvSjI3+/nrX80k9MRTkIw5LMIESfTktM
oHKmIXeYcf1yymepuwFmjEgvQ/hp0P5YWsXX3xhE+OCEoaby0AQ6FRHtSJq2y8E=
=qNAo
-----END PGP SIGNATURE-----
Nigel Kersten
--
Nigel Kersten
Product Manager, Puppet Labs
Brian Gupta
It frightens me a bit that I think the "correct" solution, will be to replicate what the distros are doing in Puppetforge. Basically turning puppetforge into a massive cross distro metadata repo, with very strict contribution standards and rules. This would involve strong rules for curated modules that would require manpower to vet (and to contribute the modules).
That leaves us what to do about "defined". (Or was that a different thread?) In the case of defined, my group has only ever used it once in almost 4 years, but it seems from the discussions that there are others still using it. Maybe the answer is provide a real alternative first, and then go about deprecating it? We wouldn't miss it, but I could see the challenges of rewriting a codebase that depends on it, and I wouldn't want that rewrite enforced on me, without a solid alternative.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Jeff McCune
<felix...@alumni.tu-berlin.de> wrote:
> Jeff has made a strong point against using virtual resources in modules
> at all, causing me to shift my own views as well.
> If I understand him correctly, one of the chief problems is the high
> probability of accidental collection/realisation of such resources by
> the end user's manifest.
Yep, that's it.
-Jeff
Brian Gupta
I just wanted to add, if we do go this route, we should work to support private "forges" (module repos) as well.
Brian
Felix Frank
On 01/28/2012 04:35 PM, Trevor Vaughan wrote:
> Drawbacks:
>
> * Requires the user to have an explicit working knowledge of all
modules and namespaces
> * Adds a lot of random logic to the code (unless it becomes a
metaparam of some sort)
You skipped the most important drawback: Commitment to parameterized
classes. The fact that there can be only one place that includes those
classes, and that this singular place must have the whole picture of
what requirements are met, is conceivably a show stopper from my point
of view.
This will work for people that have a functional ENC, I guess, but
should that be a requirement for using Forge modules?
Furthermore, how can modules hope to ever interoperate like this? If all
module classes get parameterized, it will be outright impossible for one
module to ever include another module's classes.
Say module A includes class B::C. As soon as a user installs module A in
addition to B, they have to clean their manifests of inclusions of B::C.
On 01/29/2012 07:39 AM, Brian Gupta wrote:
> It frightens me a bit that I think the "correct" solution, will be to
> replicate what the distros are doing in Puppetforge. Basically turning
> puppetforge into a massive cross distro metadata repo, with very strict
> contribution standards and rules. This would involve strong rules for
> curated modules that would require manpower to vet (and to
> contribute the modules).
I honestly don't see the problem. Imagine CPAN was limited to downloads
of tarballs from the website (or even souce control checkouts). I
disbelieve it would be as significant today as it has become.
The same goes for Ruby Gems and all such systems.
As this seems to be a recurring theme: Am I wrong to compare these to
the Forge?
Sincerely,
Felix
Walter Heck
Brian: a forge with one module for every purpose, supported maybe by
library modules (like linux) would be the best. It would mean
consolidating the 17 ssh modules that exist now into one all
encompassing ssh module. In the case of SSH that is totally doable,
but consider the 300 gazillion different use cases of apache and
things get interesting. I think this would mean we'd have appointed
maintainers for each module, and if possible a puppetlabs provided
jenkins install (with community provided jenkins slaves?) that would
do the CI testing every night. That would then make it possible to
have an auto-updated overview for each module of the latest supported
OS's for puppet, and wether that module passes tests for that OS.
This seems like a massive undertaking from where we are now, but it
would in the end make all of our lives a ton easier (one trusted
source for good high quality modules) and reduce the 'problem' of
inter-module dependencies to a minimum. Of course it still exists for
in-house applications that are being puppetised, but it would already
mean the world if they would be able to depend on what the public
trusted modules define.
I personally like the way the drupal module projects work: anyone can
start a project, but they are all hosted on the drupal.org site within
drupal.org version control, and they have teams of code reviewers
maintaining integrity of the module base that lives on drupal.org.
cheers,
Walter
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>
--
Trevor Vaughan
Hash: SHA1
It is annoying to have everything in a single place that defines the state of your nodes but, as you point out, this seems to be the model if you're
using an ENC and that seems to be the recommended practice across the mailing list for any sort of scale.
But, you don't need a functional ENC to make this work, you simply need to have everything defined at the top level whether it be via node, higher
level class, or ENC.
The main issue here seems to be modules that are trying to be "too helpful" from my reading of the mailing list.
It seems that many would like this to be an anti-pattern:
class foo {
if not defined(Class['bar']) {
include 'bar'
bar::baz { ... }
}
}
include 'foo'
Instead, you should be less helpful, and do the following:
include 'foo'
include 'bar'
bar::baz { ... }
So, instead of doing something like, say, setting up IPTables in your module (thus creating a cross-module dependency), you should do all of this in
one monolithic place at the node level or a higher level aggregation class level.
While this keeps your modules clean, it seems like a lot more effort to maintain since the module for nginx should really know what ports it's using
and know how to set up its own firewall rules.
So, the tradeoff is an ENC vs. a large collection of cluttered classes at the top level to make sure you don't have cross-module dependencies.
I'm not sure if either is better (or if either is any good at all) but they're both functional.
The ability to tag modules as requiring other modules of a particular version (ala CPAN, Gems, everything else....) would solve this issue as Puppet
would be able to check to make sure that you have the correct version of the modules installed prior to compiling the catalog.
Trevor
- --
Trevor Vaughan
Vice President, Onyx Point, Inc.
email: tvau...@onyxpoint.com
phone: 410-541-ONYX (6699)
pgp: 0x6C701E94
- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEcBAEBAgAGBQJPJoUxAAoJECNCGV1OLcyp7igH/0rroAjC8Ewc9Aw2bdE7gO0N
0KfvzYCTZtJLFTBeNIAErliWd9iR5W84H0j8KJGjPg18qcRcDHjC/hnf5+GV8lIS
6kG3EgwYwyDg8Xc0qAbWubJv7bJ29X4Fc8CCHkq13CkXFM/OqnKpUbXA6X6+o5a/
Hv5Z6WXQjPC7uCupwyqktkjj5sjwvdgniSvKsj6EK3bhGRyMsvJAzmDjucwcRNsM
vz6IG05aFJrYTUp0rZzTJf/HjIPgmD90puoXXSa/RVQnsb3WSw0AwYe1jBAHWap4
pDw+F2qrMdwc9XgQv4ZFFNp/A1OCFh21uW3B1D7XjM+U3QRpmXTKhX71lcqbX08=
=XRv1
-----END PGP SIGNATURE-----
jcbollinger
On Jan 30, 2:51 am, Felix Frank <felix.fr...@alumni.tu-berlin.de>
wrote:
>
> On 01/28/2012 04:35 PM, Trevor Vaughan wrote:> Drawbacks:
>
> > * Requires the user to have an explicit working knowledge of all
>
> modules and namespaces> * Adds a lot of random logic to the code (unless it becomes a
>
> metaparam of some sort)
>
> You skipped the most important drawback: Commitment to parameterized
> classes. The fact that there can be only one place that includes those
> classes, and that this singular place must have the whole picture of
> what requirements are met, is conceivably a show stopper from my point
> of view.
> This will work for people that have a functional ENC, I guess, but
> should that be a requirement for using Forge modules?
>
> Furthermore, how can modules hope to ever interoperate like this? If all
> module classes get parameterized, it will be outright impossible for one
> module to ever include another module's classes.
> Say module A includes class B::C. As soon as a user installs module A in
> addition to B, they have to clean their manifests of inclusions of B::C.
requirements, but it depends on classes being *un*parameterized to
apply those requirements -- catch-22. There is a solution to that
particular problem, however: define requirements via external
[meta]data instead of via parameters.
That's not to say I like the proposal, however. I don't. In fact I
think it's completely inside-out. Classes should *know* what other
classes they depend upon; they should not need to be told.
Moreover, I don't see what is gained by passing requirements as
parameters rather than, say, having the user separately declare
optional classes that carry the same information. For example,
instead of a structure requiring this:
class { 'foo': reqs => ['bar'] }
it should be possible to structure a module's classes so that you can
do this instead:
include 'foo::bar'
include 'foo'
(perhaps in the opposite order).
> On 01/29/2012 07:39 AM, Brian Gupta wrote:
>
> > It frightens me a bit that I think the "correct" solution, will be to
> > replicate what the distros are doing in Puppetforge. Basically turning
> > puppetforge into a massive cross distro metadata repo, with very strict
> > contribution standards and rules. This would involve strong rules for
> > curated modules that would require manpower to vet (and to
> > contribute the modules).
>
> I honestly don't see the problem. Imagine CPAN was limited to downloads
> of tarballs from the website (or even souce control checkouts). I
> disbelieve it would be as significant today as it has become.
> The same goes for Ruby Gems and all such systems.
>
> As this seems to be a recurring theme: Am I wrong to compare these to
> the Forge?
On one hand, clearly there are structural similarities between, say,
Puppet modules and Perl modules. The Forge serves a similar role for
the former to the role CPAN serves for the latter. I can totally see
the usefulness of tools for automatically installing and removing
Forged Puppet modules.
On the other hand, there are several problems with adopting such an
approach to solve the Puppet module dependency issue, among them:
1) The Forge doesn't address the problem AT ALL as it now stands. The
relevant resource that package management systems offer is not the
packages themselves (which the Forge has), but rather the metadata
database that each system relies on. Defining, building, and
maintaining such a database would be a massive and ongoing endeavor,
quite separate from developing and managing the Puppet and module
codebases.
2) A package management system would only work around the problem, not
solve it, and then only among modules from the same repository.
3) In fact, it would not really be the package management system
itself that made anything work. Instead, authors of Forge modules
would still need to find some way to make their modules interoperate,
and Forge curators would need somehow to ensure that they did. All
the management software would really contribute is a dependency-aware
ability to automatically install/remove packages. The system overall
would provide a promise that the particular modules it contains all do
work together, but that's altogether different from a means by which
anyone -- the Forge or otherwise -- can fulfill such a promise.
John
Nigel Kersten
Nigel,
It frightens me a bit that I think the "correct" solution, will be to replicate what the distros are doing in Puppetforge. Basically turning puppetforge into a massive cross distro metadata repo, with very strict contribution standards and rules. This would involve strong rules for curated modules that would require manpower to vet (and to contribute the modules).
Nick
>> Anyway, I need to get back to work, I'll try to say more in a later email.
>
>
> That would help me to determine what I think about the idea. As it
> is, I suspect I don't quite understand what you are hoping to
> accomplish with it.
Ok, here's a couple of examples. Apologies for the length.
1. Packages
Let's say I'm writing a module called foo.
It uses a package called 'libfoo'.
Therefore I want to make sure this package is installed before the module's
configuration is applied. I use the pattern where I define separate classes to
handle install and config, and chain them together:
class foo::install {
# Make sure libfoo is installed
package { 'libfoo': ensure => present }
}
class foo::config {
# adjusts foo.conf
# ....
}
class foo {
include foo::install
include foo::config
Class[foo::install] -> Class[foo::config]
}
Bingo! Works fine. Until I want to add another 3rd party module which needs
libfoo. Now I have to move the package declaration out to a shared class and
update both modules to fit: maybe the other module wants a specific version,
other install_options, or whatever.
Sounds simple in a short example like this, except, that:
- I've been forced to customise what should be an 'off the shelf' module,
- I have to figure out what the shared class/module should say
- and fix an arbitrary name for it
- I've therefore hard-wired strong coupling in both to my shared class
- I've added potential for refactoring breakage
- and more of the same sort of problems when scaling up
Genuinely reusable modules seem nearly impossible to write as it stands. If I
want to publish my module on Puppet Forge, then the shared::libfoo module must
be published too. Except it might not agree with other published modules' ideas
about what a shared::libfoo should declare, or even be called, and so it is not
typically re-usable without refactoring.
Or, I don't publish it, leave a dangling dependency on a class called
shared::libfoo. I am still hard-wiring a name, but not what it does. I have to
tell users the name the module they must define, and a list of requirements to
put in it on our behalf.
Or, I just don't define anything about libfoo except in the documentation.
Which seems the most practical thing to do, assuming that things break
intelligibly if libfoo is absent, but this really amounts to giving up and
moving on.
Or, maybe there currently are better ways than all the above - but if so I'm
unclear what.
Now imagine we could simply assert a requirement on a package, without actually
managing it. For the sake of this example, I'll invent a syntax for
"Assertions" similar to that used for Virtual Resources which use an '@' sigil
prefix. I'll arbitrarily use a '+' instead:
+package { 'libfoo': ensure => present }
This just means "ensure libfoo is installed". It changes nothing about the
target system. It does not mean "go and install the 'libfoo' package". It does
not mean "I own the 'libfoo' package resource, and I'll fight anyone who tries
to say otherwise".
Therefore, this type of assertion can be repeated multiple times in different
modules. Possibly in slightly different ways - with extra attributes, etc.
Puppet should just check they are all consistent, and fail if they aren't, or if
the net requirements are not met. I don't know enough about Puppet internals to
say for sure, but as described in my previous email: because the Assertion
changes nothing, I hope this would be relatively easy to implement.
Now I can write my module using an Assertion instead:
class foo::install {
# Make sure libfoo is installed
+package { 'libfoo': ensure => present }
# ...
}
...and I no longer have to find the common ground between modules which use
libfoo, and/or modify the modules to use the shared declaration.
Also, we have lost an explicit dependency on a shared module arbitrarily called
'shared::libfoo' which merely declared:
package { 'libfoo': ensure => present }
So I no longer need to publish this shared module and either dictate to, or
negotiate with with potential users of my module about the intersection of our
requirements. Nor do I need to omit this requirement entirely (which might be
the only practical alternative).
Yet I am still checking the prerequisites are there.
Of course, I may still have to create a package which actually does the
appropriate package install. Or maybe not? Perhaps my provisioning system does
that for me, and I can skip that step? Either way, the knowledge that my system
is still checking the prerequisites are there.
If my prerequisites are missing, I would hope Puppet would give helpful errors
showing what needed what, and I can add a declaration to install the right
packages in a top-level "glue" class. But means we can avoid hard-wiring
arbitrary module names into the component modules.
In summary, this would be simpler and more effective than any existing Puppet
pattern I know about.
2. Creating user accounts
Another example, which was the topic of my earlier post "constraint checking".
Say I want to create a custom resource which sets up user accounts for me in a
manner of my choice.
define user_account(
$name,
$home,
$shell,
$passwd,
$uid,
$gid,
$groups,
) {
# I want to validate $home, $shell, $groups exist and are usable...
# This is a classic case where one is tempted to use this anti-pattern
# to define something usable if it doesn't exist:
if !define(Group[$gid]) {
group { $gid: ensure => present }
}
# If I can't do that, perhaps I can just depend and hope it's picked up
# elsewhere?
require File[$shell]
# ... except this can't say anything about $shell, like
# "it must be executable".
# .... do other stuff here ....
# Now define the user resource which creates the user
# (but in my tests, does not seem to check the requirements
# exist.)
user {$name:
ensure => present,
home => $home,
shell => $shell,
passwd =>
uid => $uid,
gid => $gid,
groups => $groups,
}
}
Imagine we could use Assertions as described above. Validating the parameters
is now straightforward:
define user_account(
$name,
$home,
$shell,
$passwd,
$uid,
$gid,
$groups,
) {
# Ensure $home, $shell, gid, $groups exist
# We want to own this one
group { $gid: ensure => present }
# These are shared
+group { $groups: ensure => present }
# This is shared
+file { $shell:
ensure => present,
mode => '0744', # ...if only I could merely say "a+x" here
}
# ... do other stuff ...
# Now define the resource:
user {$name:
ensure => present,
home => $home,
shell => $shell,
passwd =>
uid => $uid,
gid => $gid,
groups => $groups,
}
}
And as above:
- only User[$name] or Group[$gid] will ever conflict (which is what we want)
- there are no shared module dependencies forced into existence
- we are not risking silly mistakes like shell => '/bin/bqsh' or
groups => ['weel']
There's more I could say, but I hope this gives the basic idea.
Cheers,
N
Felix Frank
thanks for your elaborate design sketch.
Sorry for limiting my quote severely.
On 01/30/2012 06:28 PM, Nick wrote:
> +package { 'libfoo': ensure => present }
Is this different from John's "constraint" proposal?
To me this didn't become clear: Does the manifest still need to declare
an actual package { "libfoo" } somewhere, or is this implied by at least
one assertion regarding any of its parameters?
Of the latter is the case, then this is not different from just allowing
puppet to consume multiple declarations of the same resource, along with
all the oft-discussed difficulties.
If instead, there still is that one central resource declaration
somewhere, I'm pretty sure this is the same as constraints.
Which is probably a really neat idea.
Cheers,
Felix
Nick
> Hi,
>
> thanks for your elaborate design sketch.
>
> Sorry for limiting my quote severely.
>
> On 01/30/2012 06:28 PM, Nick wrote:
>> +package { 'libfoo': ensure => present }
>
> Is this different from John's "constraint" proposal?
It did sound similar, yes - but unless I misunderstand it, not identical. For
example, I don't understand how Constraints would avoid the problems with
unifying resources that Nan mentioned.
John's example appeared to be wrapping an existing Resource with something which
puts constraints on it, i.e. a layer on top of "Resources". It implies a regular
Resource to wrap somewhere.
Whereas what I had in mind was something which in principle at least, was more
basic than a Resource. With an "Assertion" there is nothing being managed, or
mutated, by definition. It defines conditions on a resource (lower-case 'r')
which can be checked, and merged, but doesn't imply that any Resource
(upper-case 'R') need to be declared. It's quite possible that one wouldn't
bother, if you don't need to manage or mutate anything.
So Resources (upper case 'R') could be thought of as extensions to Assertions
which also supply rules to mutate a system's state, should the conditions of the
Assertion not be met, so that the conditions *are* met.
> To me this didn't become clear: Does the manifest still need to declare
> an actual package { "libfoo" } somewhere, or is this implied by at least
> one assertion regarding any of its parameters?
To be explicit: if an Assertion "+package { libfoo: }" is declared, it just
means "libfoo must be installed for this manifest to work". I don't think it
needs to mandate a declaration of a full-blown "package { libfoo: }" somewhere.
In fact, I can probably imagine circumstances when something might be invoked
which indirectly takes care of the "libfoo" package (or file, or whatever) - and
then being forced to manage the "libfoo" package in Puppet just because you want
to assert its presence could be a liability.
N
Felix Frank
On 01/30/2012 10:28 PM, Nick wrote:
> It did sound similar, yes - but unless I misunderstand it, not identical. For
> example, I don't understand how Constraints would avoid the problems with
> unifying resources that Nan mentioned.
as far as I understand, there is no need to merge anything. The catalog
will or will not contain a certain resource. If existing, the resource
will have a certain set of parameteres and metaparameters. Each
constraint can be individually compared to this state. If a constrained
resource does not exist, or any of its (meta)parameters deviate from
what the constraint defines, the catalog is no longer valid.
The beauty of this design is that the language is very expressive and
all validation can be done by the master.
Err, right, John? :-)
> John's example appeared to be wrapping an existing Resource with something which
> puts constraints on it, i.e. a layer on top of "Resources". It implies a regular
> Resource to wrap somewhere.
>
> Whereas what I had in mind was something which in principle at least, was more
> basic than a Resource. With an "Assertion" there is nothing being managed, or
> mutated, by definition. It defines conditions on a resource (lower-case 'r')
> which can be checked, and merged, but doesn't imply that any Resource
> (upper-case 'R') need to be declared. It's quite possible that one wouldn't
> bother, if you don't need to manage or mutate anything.
Ah, so you'd have the agent verify if all assertions (which need to
appear as first-class citizens in the catalog) hold true, and otherwise
fail the catalog?
That strikes me as very elegant indeed.
How will situations be handled where assertions won't hold true until
parts of the catalog have been applied?
> So Resources (upper case 'R') could be thought of as extensions to Assertions
> which also supply rules to mutate a system's state, should the conditions of the
> Assertion not be met, so that the conditions *are* met.
Let's not alienate the community by declassing the proven and beloved
Resources ;-) but I've got to say, this idea does hold merit.
So does the constraint idea. Something tells me that both might be of
benefit, but I'm afraid of years of user confusion to come when everyone
is tasked with understanding the difference between the two and to
decide when to use which.
If we need to take a pick, there's two things I'd have to say for
constraints:
1. They're more closely tailored to the problem at hand.
2. They're stronger in chorus with what puppet is today.
Assertions would probably widen the borders of what's possible with
puppet (and how easy it is), and they would allow/require us to part
with some paradigms. I'm torn whether we want this sooner than seeing
the multiple declaration problem sorted out in a less intrusive way.
Cheers,
Felix
Nick
> Ah, so you'd have the agent verify if all assertions (which need to
> appear as first-class citizens in the catalog) hold true, and otherwise
> fail the catalog?
>
> That strikes me as very elegant indeed.
>
> How will situations be handled where assertions won't hold true until
> parts of the catalog have been applied?
I'm not familiar enough with Puppet's internals to answer that very confidently.
My guess is that one might be able to express requires/before relationships
between Assertions and Resources in order to enforce things like this.
The main implication of that would be to restrict the freedom to assume that
assertions can be applied in any order, because the agent's application of the
catalog would need to be split into a sequence of "assertion" and "mutation"
steps. An Assertion must then not be moved outside the part of the sequence
assigned to it.
N
jcbollinger
On Jan 31, 3:01 am, Felix Frank <felix.fr...@alumni.tu-berlin.de>
wrote:
>
> On 01/30/2012 10:28 PM, Nick wrote:
>
> > It did sound similar, yes - but unless I misunderstand it, not identical. For
> > example, I don't understand how Constraints would avoid the problems with
> > unifying resources that Nan mentioned.
>
> as far as I understand, there is no need to merge anything. The catalog
> will or will not contain a certain resource.
instances. Instead, they use resource type and title, or perhaps a
resource reference, to identify a resource to which they apply.
Traditional Puppet rules are in effect -- there is never more than one
Resource per catalog for each (type, title) pair.
> If existing, the resource
> will have a certain set of parameteres and metaparameters. Each
> constraint can be individually compared to this state. If a constrained
> resource does not exist, or any of its (meta)parameters deviate from
> what the constraint defines, the catalog is no longer valid.
leave it there, and that would be a good and useful solution.
Something like that could even be implemented in 2.7, as it introduces
no incompatibility whatever.
I am proposing that constraints might go even further, however, in one
or both of these ways:
1) Where the resource definition to which one or more constraints
apply does not explicitly specify a value for a constrained parameter
or property, applicable constraints are used to *choose* a value
(rather than allowing that property to take a default value). This
would help with the problem of writing compatible resource definitions
in the first place. Example:
Somewhere:
----
package { 'foo':
provider => 'yum'
}
Somewhere else:
----
constraint { 'my-package-foo-ensure':
resource => Package['foo'],
property => 'ensure',
forbidden_values => ['absent', 'purged'],
preferred_values => 'latest'
}
Yet another other place:
----
constraint { 'your-package-foo-provider':
resource => Package['foo'],
property => 'provider',
allowed_values => 'yum',
}
The definition of Package['foo'] explicitly specifies the 'provider'
property, so Constraint['your-package-foo-provider'] only validates
it, failing catalog compilation if the value does not satisfy the
constraint. The definition does not explicitly specify a value for
the 'ensure' property, however, so Puppet *chooses* one that is
mutually agreeable to the relevant constraints. In this case that
would work out to 'latest'. If there is no mutually agreeable value
then catalog compilation fails.
2) Extending (1), if constraints are declared for a resource that is
not otherwise itself declared for a node, then Puppet synthesizes a
declaration, using the applicable contraints to set values for all
constrained properties. This could be considered a create_resources()
on steroids.
These would make Constraints much more useful and powerful because
they lessen the burden on module users to write suitable resource
declarations. Instead of merely testing after the fact whether an
appropriate Resource was declared, item (1) would allow modules to
*collaborate* to ensure that suitable Resources are declared, without
requiring those modules to have any knowledge of one another. Item
(2) would further reduce the burden on module users by alleviating any
need for them to explicitly declare modules' needed resources
anywhere, though they could provide such declarations if they wished,
subject to the same rules that apply already.
> The beauty of this design is that the language is very expressive and
> all validation can be done by the master.
>
> Err, right, John? :-)
compliant with current DSL syntax. I'm not sure how magical the
implementation of the Constraint [meta]resource type would need to be,
but at least in principle I think it fits very cleanly into Puppet's
model. Furthermore, because all the work is done on the master,
*nothing* about the agent or any provider needs to change. Probably
no resource type implementation needs to change, either.
> > John's example appeared to be wrapping an existing Resource with something which
> > puts constraints on it, i.e. a layer on top of "Resources". It implies a regular
> > Resource to wrap somewhere.
>
> > Whereas what I had in mind was something which in principle at least, was more
> > basic than a Resource. With an "Assertion" there is nothing being managed, or
> > mutated, by definition. It defines conditions on a resource (lower-case 'r')
> > which can be checked, and merged, but doesn't imply that any Resource
> > (upper-case 'R') need to be declared. It's quite possible that one wouldn't
> > bother, if you don't need to manage or mutate anything.
>
> Ah, so you'd have the agent verify if all assertions (which need to
> appear as first-class citizens in the catalog) hold true, and otherwise
> fail the catalog?
understand that proposal:
1) Assertions are tested by the agent (or else they do nothing that
constraints do not also do), so they need to be represented in the
catalog. This makes them no lighter-weight than ordinary Resources as
far as I can tell.
2) Supposing that Assertions are indeed tested by the agent, they
defer catalog failures to the agent, which makes module interoperation
(the issue we're trying do deal with) harder to test.
3) Assertions leave the burden on module users to provide resource
declarations that are agreeable to all modules.
> That strikes me as very elegant indeed.
acknowledge that there may be cases where one wants the agent to
perform a test without changing anything. At present, one would
probably use an Exec for the purpose. I tend to think, however, that
such cases usually reflect module design shortcomings.
The Puppet paradigm is to tell the agent how a node should look, then
let the agent make it so. Why, then, would it be desirable to give
the agent contradictory node descriptions? Why would it be desirable
to give the agent a node description, but not allow it to make the
node meet the description? How would either capability further the
goal of module interoperability? Especially, why should an
interoperability solution that relies on such behaviors be preferred
to one that does not?
Assertions would be an interoperability aid for one of the same
reasons that Constraints would be: they would allow modules to express
resource requirements without actually declaring resources. This is a
better solution than merging multiple Resource declarations would be
because modules' requirements are expressed precisely, and also
because there is no change to the invariants of Puppet's data model.
In these respects, however, Assertions offer nothing that Constraints
do not also offer.
> How will situations be handled where assertions won't hold true until
> parts of the catalog have been applied?
relationships to adequately address that issue. At this point they
don't look lightweight at all to me; rather they look like a full-
blown new [meta]resource type (as Constraints also would be).
Implementation of Assertions separate from Resources, as Nick
suggests, would be tricky. The Assertion provider would need to
somehow hook into all other providers, including custom ones, to test
whether assertions were satisfied.
> > So Resources (upper case 'R') could be thought of as extensions to Assertions
> > which also supply rules to mutate a system's state, should the conditions of the
> > Assertion not be met, so that the conditions *are* met.
>
> Let's not alienate the community by declassing the proven and beloved
> Resources ;-) but I've got to say, this idea does hold merit.
Puppet is to flip that around: assertion nature could be an attribute
of (some) Resource instances. Consider its similarities to the
effects of the 'audit' metaparameter, for instance. Only in that way
can I reconcile Nick's characterization of Assertions as "lightweight"
and "low-level" with the apparent design and implementation
requirements.
> So does the constraint idea. Something tells me that both might be of
> benefit, but I'm afraid of years of user confusion to come when everyone
> is tasked with understanding the difference between the two and to
> decide when to use which.
>
> If we need to take a pick, there's two things I'd have to say for
> constraints:
> 1. They're more closely tailored to the problem at hand.
> 2. They're stronger in chorus with what puppet is today.
>
> Assertions would probably widen the borders of what's possible with
> puppet (and how easy it is), and they would allow/require us to part
> with some paradigms. I'm torn whether we want this sooner than seeing
> the multiple declaration problem sorted out in a less intrusive way.
Constraints, but Assertions (as I understand them) are not it. I can,
however, imagine using Assertions similarly to how I use the like-
named features of C and Java -- to validate assumptions and internal
consistency in catalogs.
John