The handy Grail of Modules Standards

[Alessandro Franceschi]

Dear all,

I'd like to submit to the list's attention this post about Puppet Modules Standard naming conventions:
http://www.example42.com/?q=The_handy_Grail_of_Modules_Standards 

and  this working draft of a proposal for a version 0.0.1 of them:

https://docs.google.com/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA/edit?usp=sharing

The topic is not new, and on this list there have already been various discussions on how to manage modules' interoperability ( for example this interesting one on cross-modules dependencies: https://groups.google.com/forum/?fromgroups#!searchin/puppet-users/modules$20standards|sort:date/puppet-users/Fvl0aOe4RPE) .

Along with the current discussions on the topic and the Puppet evolution, I think that few simple *suggested* naming conventions on common parameters for classes and defines would benefit a lot the module's ecosystem with a small effort.

I hope the discussion will have a real follow up, this time, and possibly move under the more authoritative PuppetLabs umbrella, for the moment is just a personal proposal, even if shared, at least in the intentions, with many modules' authors and Puppetteers.

You can actively contribute to the proposed naming standards on the above Google document (let me know your gmail account) and you are invited to comment here what do you think of this (not new) idea: if you think that starting from simple naming convention is a step to be done, what's your general idea on the proposed naming standards, if you have better ideas on naming and approach. 

Let's grab this handy Grail :-)

Alessandro Franceschi

Example42

[Matthias Saou]

Hi,

First off... is these some pun I'm not getting, or was it supposed to
be "Holy Grail"? Just wondering :-)

Also, when I read "Puppet Modules Standard naming conventions", I
thought it was just about module names, but it seems to be about
parameter naming inside all of the module resources.

After reading the shared Google document, I got reminded a lot of
what I've seen happen with RPM packaging over the years. Lot of
similarities, since both Puppet and RPM are tools which give a lot of
freedom in the way things get implemented.

Think of GNU/Linux distributions such as Fedora, RHEL, Mandriva, SuSE,
etc. which all use RPM packages as their building blocks : They are not
to be considered compatible for various reasons, yet they could have
been in theory.

With Puppet, each author is currently like each distribution using RPM :
Doing things in a given ecosystem, with implementation choices being
made, all of which can easily differ enough to make modules from
different authors incompatible.

Now the problem that I see here is that with RPM things have never been
able to converge on a global scale. There isn't one single right answer,
and it has never been the tool's goal to enforce how it's being used,
especially when it's purely cosmetic or related to details that many
find irrelevant.

Where I'm trying to get at is that I see Puppet as being similar in the
fact that it doesn't try to enforce any high-level cosmetic choices,
such as parameter names, and I think it's the right behavior for the
basic tool itself.

From there, I'm all for trying to "standardize" something, but that's
actually much harder than it seems, and won't work unless there is some
enforcing being done at some point. One possibility would be to have a
review-based approval process for forge modules, where guidelines would
have to be enforced before a module gets published. Of course, that's a
lot of time and resources, for what boils down to being considered
"boring work" by most. It's what works for many GNU/Linux distributions
(Debian, Fedora, etc.).

Just my 2¢ ;-)

Matthias

--
Matthias Saou ██ ██
██ ██
Web: http://matthias.saou.eu/ ██████████████
Mail/XMPP: matt...@saou.eu ████ ██████ ████
██████████████████████
GPG: 4096R/E755CC63 ██ ██████████████ ██
8D91 7E2E F048 9C9C 46AF ██ ██ ██ ██
21A9 7A51 7B82 E755 CC63 ████ ████

[Alessandro Franceschi]

On Monday, June 17, 2013 1:17:45 PM UTC+2, Matthias Saou wrote:

Hi,

First off... is these some pun I'm not getting, or was it supposed to
be "Holy Grail"? Just wondering :-)

It's holy but is also handy, as I think it's just at hands reach.

Thanks for your opinion, even if I don't fully agree with it.

Puppet is a language and so people do the same things in different ways, and they all work and do what they are supposed to do.

But if we think about modules REUSABILITY and INTEROPERABILITY some patterns have to be followed.
Some of the parameters described in the document are somehow REQUIRED, IMHO, if you want to make a really reusable module (for example the ones that let you decide how to manage your configuration files... if you enforce a logic in a module or a specific template and don't allow override by users, then you are not making a reusable module, so for example a parameter like "template" is just needed).
So, since, at least some of, these parameters are needed for a reusable module  it's just a matter of defining few naming conventions (and managing external modules dependencies in a sane way) to make different modules happily live better together.

Note, I don't say that ALL the modules should have ALL these parameters, I'd consider these Standard Namings as suggestions which people may decide to follow or not (somehow similar to the Code Style suggestions, which leverage the style of the Puppet code and have found tools like puppet-lint to validate them). Once enough good and prominent Puppet modules will follow these naming conventions, it will be easier for people to switch modules, integrate the best ones from different sources (without forking them) , use these parameters from a WEB interface, a a standard framework from smoke testing and have the benefits which are better described in the blog post.

Note also that these proposals are based on the current Puppet language specifications, I want to start from what can be used now, with an eye on the evolution on Puppet, but with still feet on the ground: nothing new or to invent, just few basic naming convention to agree upon and suggest.

I still think that this is at hands reach :-)

[Matthias Saou]

> agree upon and *suggest*.

>
> I still think that this is at hands reach :-)

This is definitely a good initiative, what I'm just saying is that
you've opened a can of worms :-)

The initial step of creating common guidelines for parameter names is
nice, as it can create some consistency across modules, and ease work
sharing as well as lower the learning curve for people using 3rd party
modules. But it would need to be official (in the puppet documentation
as best practices, for instance) and/or enforced on the forge, to
become really useful.

And after that, things quickly get exponentially complex IMHO. A few
examples from the top of my head :

* Naming the modules themselves.
* Naming the classes and definitions inside the modules.
* Multiple modules requiring the same packages (If my module needs
rsync, yours too, where do we put the common virtual resource?).
* The use of author-specific common modules (I don't like taking a
johndoe/apache module and noticing I then need johndoe/common).

But don't get me wrong, I like where this is headed, and will
participate as much as I can.

[Alessandro Franceschi]

Lol, we have to do that sooner or later, I think :-D
And the sooner, the better.

 

The initial step of creating common guidelines for parameter names is
nice, as it can create some consistency across modules, and ease work
sharing as well as lower the learning curve for people using 3rd party
modules. But it would need to be official (in the puppet documentation
as best practices, for instance) and/or enforced on the forge, to
become really useful.

I definitively agree.
This is something that should be endorsed if not managed directly by PuppetLabs.
Anyway I would not "enforce" the use of standard params, but somehow "certify" the modules that provide them (eventually defining different "levels" of standard params coverage) so that you know you can use them with established patterns (and in the future maybe test them with standard procedures and integrate them in an ENC that explicitly supports and exposes these standard parameters)

 

And after that, things quickly get exponentially complex IMHO. A few
examples from the top of my head :

 * Naming the modules themselves.

Right, even if, besides few cases with somehow flurry namings (apache or httpd? ssh or openssh?), generally the name of the module is already quite standard.

 

 * Naming the classes and definitions inside the modules.

 * Multiple modules requiring the same packages (If my module needs
   rsync, yours too, where do we put the common virtual resource?).

This is a wide and tricky issue. My naif approach is the definition of a dependency_class parameter where external resources like rsync are managed (and you can provide a custom dependency_class where you manage the required resources in the way you want). For the installation of simple packages I'd currently use/recommend the not perfect "if ! defined" approach (always in the dependency_class).
When the language will provide a smarter solution, we can follow it, some past discussion on possible solutions there has been in the list, in order to work they definitively have to be shared by the modules (so they fit well in the "standard module" pattern). Some of them required enhancements to the DSL some not, being a Puppet user I prefer to stick to current language syntax.

 

 * The use of author-specific common modules (I don't like taking a
   johndoe/apache module and noticing I then need johndoe/common).

I'd try to leverage modules on stdlib as much as possible, but at the same time some "local common" modules are sometimes needed.
It doesn't harm to have them in your modulepath, as long as there are no naming conflicts.

 

But don't get me wrong, I like where this is headed, and will
participate as much as I can.

Cool. Actually I found your proposals on the service management parameters, on the Google Doc draft, quite sane , so keep on! 

Al

[jcbollinger]

On Monday, June 17, 2013 9:32:36 AM UTC-5, Alessandro Franceschi wrote:

Puppet is a language and so people do the same things in different ways, and they all work and do what they are supposed to do.

But if we think about modules REUSABILITY and INTEROPERABILITY some patterns have to be followed.
Some of the parameters described in the document are somehow REQUIRED, IMHO, if you want to make a really reusable module (for example the ones that let you decide how to manage your configuration files... if you enforce a logic in a module or a specific template and don't allow override by users, then you are not making a reusable module, so for example a parameter like "template" is just needed).
So, since, at least some of, these parameters are needed for a reusable module  it's just a matter of defining few naming conventions (and managing external modules dependencies in a sane way) to make different modules happily live better together.

Although I agree that to be reusable, modules need to provide certain types of levers, knobs, and switches, as appropriate for their scopes, I think the case is weak for those controls needing to be called by the same names.  At best, naming conventions for such things might improve ease of (re)use for some people, but the key factor for reusability is not the names of the controls so much as their presence in the first place.

I see implications for interoperability only insomuch as one imagines facilitating one module being a drop-in replacement for another, but (1) there's a lot more to that than just common naming, so (2) that kind of interoperability is unlikely to come about except by specific intention anyway, so in that case shared naming comes out as a project requirement.  To me, that moots any general parameter-naming standard as far as interoperability goes.
 
None of that is a fundamental reason to object to the effort, but I'm not seeing any promise of significant benefit to motivate me to participate actively.

I do have a bona fide objection, however: although the effort is cast at this point as being aimed exclusively at parameter naming, by implication it also covers elements of module structure, organization, and style as well, as the things being named have to exist for the standard to be relevant.  I do understand that the intention is not to make all the standardized controls mandatory for every module, but for those that a given module does provide, even the standard's limited reusability and interoperability goals are not served if those controls are not located where the standard anticipates (which is for the most part on a single front-end class).

Personally, I would rather see a white paper explaining what kinds of controls need to be available to facilitate reusability of various kinds of modules, and, optionally, setting out one or more models of how a module can provide those controls.  Regardless of the nature of the paper's authorship, this feels like it should be a position paper, not a proposed standard.

John

[Ken Barber]

> Although I agree that to be reusable, modules need to provide certain types
> of levers, knobs, and switches, as appropriate for their scopes, I think the
> case is weak for those controls needing to be called by the same names. At
> best, naming conventions for such things might improve ease of (re)use for
> some people, but the key factor for reusability is not the names of the
> controls so much as their presence in the first place.
>
> I see implications for interoperability only insomuch as one imagines
> facilitating one module being a drop-in replacement for another, but (1)
> there's a lot more to that than just common naming, so (2) that kind of
> interoperability is unlikely to come about except by specific intention
> anyway, so in that case shared naming comes out as a project requirement.
> To me, that moots any general parameter-naming standard as far as
> interoperability goes.

I think being able to use another class in a drop-in way is not the
value I see in parameter naming 'recommendations'. I personal see
value in the ease of use more than anything, if parameters are
similarly named between classes, when you go to use them you don't
have to interrupt, double check with the docs what this class/define
uses, then modify your parameter name accordingly. Its a reduction in
surprise if anything. An example would be the 'package' parameter,
versus 'packages' ... if I didn't have to stop and check which one it
is for my XYZ class it might save time and mistakes *shrug*.

Is that valuable? Alas, I'm more of developer then user these days so
I would defer that to our users. As a developer though - I would find
it handy to have a guide for common things, I'm a pedant when it comes
to naming and if someone already came up with a name for me, I would
probably use it, presuming others have thought through any naming
consequences.

> None of that is a fundamental reason to object to the effort, but I'm not
> seeing any promise of significant benefit to motivate me to participate
> actively.
>
> I do have a bona fide objection, however: although the effort is cast at
> this point as being aimed exclusively at parameter naming, by implication it
> also covers elements of module structure, organization, and style as well,
> as the things being named have to exist for the standard to be relevant. I
> do understand that the intention is not to make all the standardized
> controls mandatory for every module, but for those that a given module does
> provide, even the standard's limited reusability and interoperability goals
> are not served if those controls are not located where the standard
> anticipates (which is for the most part on a single front-end class).

I've been pondering this situation as well. I presume in a world where
such recommendations become commonly used, the outcome would be
surprise at a missing 'recommended' parameter, then a subsequent bug
raised on the module due to its lack. This might be considered a
positive or negative. If the parameter name was named 'something else'
due to a feeling that the 'standard' is not covering a developers
needs, then this could be annoying to have that discussion _yet
again_. If however the functionality is simply missing - this becomes
a BAU patch (like the lint patches we see all the time) and probably a
positive feature request.

Of course, this all depends on how good the recommendations are.
Having looked through the document I think some of them are obvious,
and require less debate while other recommendations are less
obvious/contentious.

> Personally, I would rather see a white paper explaining what kinds of
> controls need to be available to facilitate reusability of various kinds of
> modules, and, optionally, setting out one or more models of how a module can
> provide those controls. Regardless of the nature of the paper's authorship,
> this feels like it should be a position paper, not a proposed standard.

Fair point, its a difficult document to position precisely. I guess I
foresee this as something that should be a part of a guide for writing
modules then any hard/fast rule or 'standard'.

ken.

[Alessandro Franceschi]

Some personal notes among the lines...

On Tuesday, June 18, 2013 6:35:18 PM UTC+2, Ken Barber wrote:

> Although I agree that to be reusable, modules need to provide certain types
> of levers, knobs, and switches, as appropriate for their scopes, I think the
> case is weak for those controls needing to be called by the same names.  At
> best, naming conventions for such things might improve ease of (re)use for
> some people, but the key factor for reusability is not the names of the
> controls so much as their presence in the first place.

Well on this I definitively do not agree :-)
For me a module is reusable when:
1- It supports many different OS (this is somehow implicit and does not involve naming conventions)

2- It leaves to the user freedom on how to populate and customize the provided configuration files ( I think this is the main point for most reusability cases)
3- It allows  the user to manage some behaviours of the module (has a service to be restarted after a file change? Do I want to manage a service status (at runtime or boot)
4- In (somehow extreme) cases it allows the user to customize names of package/services, paths of files and so on

5- It allows seamless addition of custom resources, not managed by the module but related to it

6- It allows the user to decide where to place his data (also this is out of naming convention scope)

Given these points, I think that some of the parameters names proposed in the draft actually DO inherently enhance a module reusability:
For point 2: source, template, options (with might make useless almost any additional application specific configuration parameter), dir_source (and  dir_* )
For point 3: status, autorestart (poor name), audits, noops

For point 4: package, service, file_path, dir_path

For point 5: my_class resources_hash

But maybe we have different semantic nuances for the term "modules' reusability".

>
> I see implications for interoperability only insomuch as one imagines
> facilitating one module being a drop-in replacement for another, but (1)
> there's a lot more to that than just common naming, so (2) that kind of
> interoperability is unlikely to come about except by specific intention
> anyway, so in that case shared naming comes out as a project requirement.
> To me, that moots any general parameter-naming standard as far as
> interoperability goes.

I agree that the interoperability part is not fully dependent on naming standards.

Aa a partial solution for this I have thought about the usage of a "dependency_class" to contain in a single, replaceable, class all the external dependencies, and eventually tweaking the Modulefile and the puppet module command to manage soft-dependencies and reduce conflicts modules Forge modules (more details on this on the blog post linked before).

I don't see it as a perfect solution but that's something that can be done now (without extra code if not for the Forge integration) and very quickly.

I think being able to use another class in a drop-in way is not the
value I see in parameter naming 'recommendations'. I personal see
value in the ease of use more than anything, if parameters are
similarly named between classes, when you go to use them you don't
have to interrupt, double check with the docs what this class/define
uses, then modify your parameter name accordingly. Its a reduction in
surprise if anything. An example would be the 'package' parameter,
versus 'packages' ... if I didn't have to stop and check which one it
is for my XYZ class it might save time and mistakes *shrug*.

Let me clarify that hardly in my dreams I could imagine seamless drop-in replacements, but naming standards+dependency_class pattern CAN make interoperability much easier.

For the other benefits, least surprise is one point, not small as it involves (let me copy and paste :-):

- Better user experience (modules are easier to use and understand) 

- Quicker and more reliable Puppet manifests development (for basic functions you can expect predictable parameters)

- More coherent and error proof Puppet setups 

but I see various other clear benefits, in the mid term:

- The possibility to have an unified approach to smoke testing of common features 

- The possibility to have web front-ends and ENC that leverage on the standardized parameters 

- Easier integration with superclasses that expose their own parameters and use different modules to build up full applications stacks or complex setups that involve multiple modules.

- A PuppetLabs and/or Community driven central repository of well tested and features rich unique Standard modules
(it might be a subset of the Forge or a set of git repos with only one module for application)

 

Is that valuable? Alas, I'm more of developer then user these days so
I would defer that to our users. As a developer though - I would find
it handy to have a guide for common things, I'm a pedant when it comes
to naming and if someone already came up with a name for me, I would
probably use it, presuming others have thought through any naming
consequences.

> None of that is a fundamental reason to object to the effort, but I'm not
> seeing any promise of significant benefit to motivate me to participate
> actively.
>
> I do have a bona fide objection, however: although the effort is cast at
> this point as being aimed exclusively at parameter naming, by implication it
> also covers elements of module structure, organization, and style as well,
> as the things being named have to exist for the standard to be relevant.  I
> do understand that the intention is not to make all the standardized
> controls mandatory for every module, but for those that a given module does
> provide, even the standard's limited reusability and interoperability goals
> are not served if those controls are not located where the standard
> anticipates (which is for the most part on a single front-end class). 

The assumption is that parameters are passed to the main module class or to single specific defines.
This is the only constraint to the module structure then you can organize the modules as you want. 

These are two sample modules that honour the same standard parameters with totally different implementations for a typical package/service/file module type:

https://github.com/example42/puppet-stdmod

https://github.com/example42/puppet-stdmodalt 

I've been pondering this situation as well. I presume in a world where
such recommendations become commonly used, the outcome would be
surprise at a missing 'recommended' parameter, then a subsequent bug
raised on the module due to its lack. This might be considered a
positive or negative. If the parameter name was named 'something else'
due to a feeling that the 'standard' is not covering a developers
needs, then this could be annoying to have that discussion _yet
again_. If however the functionality is simply missing - this becomes
a BAU patch (like the lint patches we see all the time) and probably a
positive feature request.

To ease the migration of modules to the standard namings, for overlapping parameters it could be useful to create a function and provide some sample code to manage duplicated parameters and notify the user with a deprecation notice when the "old" parameter is used 
I definitively would not throw away all the current modules work for the reckless seek of some standard fanciness.
We can think of tools, methods and suggestions to ease the migration of existing modules and to speed the creation of new standard modules (the above sample modules are made to be easily clonable and may serve as blueprint for new modules).

Of course, this all depends on how good the recommendations are.
Having looked through the document I think some of them are obvious,
and require less debate while other recommendations are less
obvious/contentious.

The intention of the shared google doc is exactly to start to enumerate what are possible sane "standard" parameters and give them a good and coherent name.
Please comment on the doc what are the params for which you'd choose a different name and, most important, what are the ones that should not be there (and obviously add some missing ones you may think about)

 

> Personally, I would rather see a white paper explaining what kinds of
> controls need to be available to facilitate reusability of various kinds of
> modules, and, optionally, setting out one or more models of how a module can
> provide those controls.  Regardless of the nature of the paper's authorship,
> this feels like it should be a position paper, not a proposed standard.

Fair point, its a difficult document to position precisely. I guess I
foresee this as something that should be a part of a guide for writing
modules then any hard/fast rule or 'standard'.

Call it position paper or draft for a proposed standard, what's important, imho, is the final goal:
some recommendations for naming conventions to use in modules (parameters) that can help in making some order in the modules' casbah while suggestion patterns that enhance reusability and may improve interoperabiity.

Since it's at least 3 years that there're some sparse discussions about modules reusability I preferred to give some solid ground upon which have a discussion, for this reason I placed the links of  the stdmod sample modules and the Google Doc.

Please consider this as a starting point to improve and eventually radically change (draft for 0.0.1 means that there's STILL to be a 0.0.1 and whatever comes after).

Thank you guys for the insights, hope to see your contributions to the Doc, both for the parts that find you skeptic and the parts you agree upon. I think it's important to do something that is well thought and shared but is even more important to start to do it, step by step, and I hope we all agree that is is something to do, sooner or later to avoid total chaos and fragmentation in the modules ecosystem.

my2c
al

[jcbollinger]

On Tuesday, June 18, 2013 1:19:32 PM UTC-5, Alessandro Franceschi wrote:

Some personal notes among the lines...

On Tuesday, June 18, 2013 6:35:18 PM UTC+2, Ken Barber wrote:

> Although I agree that to be reusable, modules need to provide certain types
> of levers, knobs, and switches, as appropriate for their scopes, I think the
> case is weak for those controls needing to be called by the same names.  At
> best, naming conventions for such things might improve ease of (re)use for
> some people, but the key factor for reusability is not the names of the
> controls so much as their presence in the first place.

Well on this I definitively do not agree :-)
For me a module is reusable when:
1- It supports many different OS (this is somehow implicit and does not involve naming conventions)

2- It leaves to the user freedom on how to populate and customize the provided configuration files ( I think this is the main point for most reusability cases)
3- It allows  the user to manage some behaviours of the module (has a service to be restarted after a file change? Do I want to manage a service status (at runtime or boot)
4- In (somehow extreme) cases it allows the user to customize names of package/services, paths of files and so on

5- It allows seamless addition of custom resources, not managed by the module but related to it

6- It allows the user to decide where to place his data (also this is out of naming convention scope)

Given these points, I think that some of the parameters names proposed in the draft actually DO inherently enhance a module reusability:
For point 2: source, template, options (with might make useless almost any additional application specific configuration parameter), dir_source (and  dir_* )
For point 3: status, autorestart (poor name), audits, noops

For point 4: package, service, file_path, dir_path

For point 5: my_class resources_hash

But maybe we have different semantic nuances for the term "modules' reusability".

Evidently so. For the most part, what you said does not conflict with what I said; you were just more specific.  Where we differ is that I am distinguishing "reusability" from ease of reuse.  I agree that all of your items 1 - 6 are factors that improve modules' reusability.  I assert, however, that the names by which those particular knobs are referenced do not constitute essential reusability factors, but rather ease of use factors.

Basically, I am drawing the line at having to modify the code to use a third-party module.  The module is "reusable" if I do not have to modify it, even if I have to study it to determine how to use it.  It is easy to use if it presents a coherent, intuitive external interface.  Almost all naming considerations fall into the latter category.

Moreover, any ease of use advantage of such a convention is limited to those users who buy in to the convention.  Such buy-in is unlikely to arise from a standard-setting effort alone.
 

I agree that the interoperability part is not fully dependent on naming standards.

Aa a partial solution for this I have thought about the usage of a "dependency_class" to contain in a single, replaceable, class all the external dependencies, and eventually tweaking the Modulefile and the puppet module command to manage soft-dependencies and reduce conflicts modules Forge modules (more details on this on the blog post linked before).

I don't see it as a perfect solution but that's something that can be done now (without extra code if not for the Forge integration) and very quickly.

I have some issues with that approach, but I'm going to disregard them for now because that topic is largely beyond the stated scope of the naming conventions and convention-drafting effort we are discussing.

 

I think being able to use another class in a drop-in way is not the
value I see in parameter naming 'recommendations'. I personal see
value in the ease of use more than anything, if parameters are
similarly named between classes, when you go to use them you don't
have to interrupt, double check with the docs what this class/define
uses, then modify your parameter name accordingly. Its a reduction in
surprise if anything. An example would be the 'package' parameter,
versus 'packages' ... if I didn't have to stop and check which one it
is for my XYZ class it might save time and mistakes *shrug*.

Let me clarify that hardly in my dreams I could imagine seamless drop-in replacements, but naming standards+dependency_class pattern CAN make interoperability much easier.

So, then, this really isn't just a question of naming things, as previously claimed?  I mean, I know I made that very observation in my previous post, but I had assumed that it was just an aspect of the proposal that you hadn't recognized and appreciated, not an intentional subterfuge.

 

For the other benefits, least surprise is one point, not small as it involves (let me copy and paste :-):

- Better user experience (modules are easier to use and understand) 

- Quicker and more reliable Puppet manifests development (for basic functions you can expect predictable parameters)

- More coherent and error proof Puppet setups 

Least surprise applies only if I have a reason to expect particular conventions to be followed.  If not, then I cannot be surprised.  In a sense, then, the opportunity for surprise is a cost of such conventions as are proposed, partially offsetting the potential ease of use advantages for modules that conform with such a convention.  And again, I do not deny that there are potential ease of use advantages associated with shared conventions, but that comes not from the conventions themselves, but rather from their broad acceptance and use.

 

but I see various other clear benefits, in the mid term:

- The possibility to have an unified approach to smoke testing of common features 

- The possibility to have web front-ends and ENC that leverage on the standardized parameters 

If there were good tools available for those applications then they might provide an impetus toward conformance with the proposed conventions.  However, I think adopting conventions on the hope that such tools will materialize in the future is unwise.  Call me "experienced", "jaded", or even "cynical", but I know of too many instances of vaporware to be swayed by promises of software that hasn't yet been written.

 

- Easier integration with superclasses that expose their own parameters and use different modules to build up full applications stacks or complex setups that involve multiple modules.

Sorry, I'm not seeing it.

 

- A PuppetLabs and/or Community driven central repository of well tested and features rich unique Standard modules
(it might be a subset of the Forge or a set of git repos with only one module for application)

Yes, the idea of a curated repository of interoperable modules has been raised before.  I think it's a fine idea, but I don't think it depends on parameter naming conventions for realization.  In fact, I don't think parameter naming conventions actually bring much to that particular table. 

 

 

> None of that is a fundamental reason to object to the effort, but I'm not
> seeing any promise of significant benefit to motivate me to participate
> actively.
>
> I do have a bona fide objection, however: although the effort is cast at
> this point as being aimed exclusively at parameter naming, by implication it
> also covers elements of module structure, organization, and style as well,
> as the things being named have to exist for the standard to be relevant.  I
> do understand that the intention is not to make all the standardized
> controls mandatory for every module, but for those that a given module does
> provide, even the standard's limited reusability and interoperability goals
> are not served if those controls are not located where the standard
> anticipates (which is for the most part on a single front-end class). 

The assumption is that parameters are passed to the main module class or to single specific defines.
This is the only constraint to the module structure then you can organize the modules as you want. 

It is a non-trivial constraint that all of a module's external interface be implemented in the form of a single class and zero or more types (and zero or more custom functions, heretofore not discussed).  That it can be expressed in one sentence does not make it less significant.  Furthermore, there is an implicit assumption even in that admission: that module interfaces will involve parameterized classes.  That in itself is probably not a big deal any longer, but some of the prospective advantages claimed seem predicated on the premise that parameterized-style class declarations will be used.  That is a big deal, especially given that a large ecosystem of mutually interoperable modules is hypothesized as a viable and likely outcome.

Most importantly, however, my central objection here is that a proposal that purports to be exclusively about naming parameters in fact has any implications at all for module organization and style.

Fair point, its a difficult document to position precisely. I guess I
foresee this as something that should be a part of a guide for writing
modules then any hard/fast rule or 'standard'.

Call it position paper or draft for a proposed standard, what's important, imho, is the final goal:

The names of things matter, as indeed this whole discussion presupposes.  How people respond to the initiative will be colored on many levels by what you call it and how you position it.  In fact, I might not have joined the discussion if the effort had been positioned as targeting a set of common conventions, but setting a standard is a rather different endeavor.

 

some recommendations for naming conventions to use in modules (parameters) that can help in making some order in the modules' casbah while suggestion patterns that enhance reusability and may improve interoperabiity.

Since it's at least 3 years that there're some sparse discussions about modules reusability I preferred to give some solid ground upon which have a discussion, for this reason I placed the links of  the stdmod sample modules and the Google Doc.

Please consider this as a starting point to improve and eventually radically change (draft for 0.0.1 means that there's STILL to be a 0.0.1 and whatever comes after).

Thank you guys for the insights, hope to see your contributions to the Doc, both for the parts that find you skeptic and the parts you agree upon. I think it's important to do something that is well thought and shared but is even more important to start to do it, step by step, and I hope we all agree that is is something to do, sooner or later to avoid total chaos and fragmentation in the modules ecosystem.

I think you have embarked on a worthy effort, but I also think your aspirations are too grand.  Build it, with as much community participation as you can reasonably accommodate, and let the result speak for itself.  If it succeeds well in everything you envision then it is likely to be well received and broadly implemented.

But do not set out with the expectation of that the result will be adopted or used otherwise, and certainly do not create a chicken-and-egg problem, where the effort's success depends on it being successful.


John

[Alessandro Franceschi]

Here I really don't follow you: if you agree that the above points improves modules' reusability, how can you say that defining naming standards for parameters that empower those points just enhance ease of use and not the same reusability of a module that implements them?

 

Basically, I am drawing the line at having to modify the code to use a third-party module.  The module is "reusable" if I do not have to modify it, even if I have to study it to determine how to use it.  It is easy to use if it presents a coherent, intuitive external interface.  Almost all naming considerations fall into the latter category.

Well at least we agree on the definition of a reusable module.

 

Moreover, any ease of use advantage of such a convention is limited to those users who buy in to the convention.  Such buy-in is unlikely to arise from a standard-setting effort alone.
 

I agree that the interoperability part is not fully dependent on naming standards.

Aa a partial solution for this I have thought about the usage of a "dependency_class" to contain in a single, replaceable, class all the external dependencies, and eventually tweaking the Modulefile and the puppet module command to manage soft-dependencies and reduce conflicts modules Forge modules (more details on this on the blog post linked before).

I don't see it as a perfect solution but that's something that can be done now (without extra code if not for the Forge integration) and very quickly.

I have some issues with that approach, but I'm going to disregard them for now because that topic is largely beyond the stated scope of the naming conventions and convention-drafting effort we are discussing.

 

I think being able to use another class in a drop-in way is not the
value I see in parameter naming 'recommendations'. I personal see
value in the ease of use more than anything, if parameters are
similarly named between classes, when you go to use them you don't
have to interrupt, double check with the docs what this class/define
uses, then modify your parameter name accordingly. Its a reduction in
surprise if anything. An example would be the 'package' parameter,
versus 'packages' ... if I didn't have to stop and check which one it
is for my XYZ class it might save time and mistakes *shrug*.

Let me clarify that hardly in my dreams I could imagine seamless drop-in replacements, but naming standards+dependency_class pattern CAN make interoperability much easier.

So, then, this really isn't just a question of naming things, as previously claimed?  I mean, I know I made that very observation in my previous post, but I had assumed that it was just an aspect of the proposal that you hadn't recognized and appreciated, not an intentional subterfuge.

Subterfuge is definitively a not appropriate and appreciated word for this case
All the points of this discussion are expressed in the quoted blog post:

http://www.example42.com/?q=The_handy_Grail_of_Modules_Standards 

and  the *draft* of a *proposal* for a *version 0.0.1* (I suppose this is something far from being presented as definitive) :

https://docs.google.com/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA/edit?usp=sharing

I can't repeat at every post or sentence the whole vision on how appropriate  module naming standards may improve modules' reusability, interoperability and ultimately quality.

 

For the other benefits, least surprise is one point, not small as it involves (let me copy and paste :-):

- Better user experience (modules are easier to use and understand) 

- Quicker and more reliable Puppet manifests development (for basic functions you can expect predictable parameters)

- More coherent and error proof Puppet setups 

Least surprise applies only if I have a reason to expect particular conventions to be followed.  If not, then I cannot be surprised.  In a sense, then, the opportunity for surprise is a cost of such conventions as are proposed, partially offsetting the potential ease of use advantages for modules that conform with such a convention.  And again, I do not deny that there are potential ease of use advantages associated with shared conventions, but that comes not from the conventions themselves, but rather from their broad acceptance and use.

Broad acceptance and use definitively will arrive if:
- Sane naming standards are proposed that effectively enhance modules' reusability
- PuppetLabs "suggests" and endorses them

Some of the conventions of Puppet' style language can be considered arbitrary, still they are receiving wide acceptance because PuppetLabs has proposed them and tools like puppet-lint (emerged later) make easier to adapt to them.

Really I can't see why something similar can't happen for modules naming standards (and yes, some naming conventions involve specific features and functionality that definitively can make a module more reusable, so it's not JUST a matter of giving names to things but also to suggest patterns and techniques that make modules better)
 

 

but I see various other clear benefits, in the mid term:

- The possibility to have an unified approach to smoke testing of common features 

- The possibility to have web front-ends and ENC that leverage on the standardized parameters 

If there were good tools available for those applications then they might provide an impetus toward conformance with the proposed conventions.  However, I think adopting conventions on the hope that such tools will materialize in the future is unwise.  Call me "experienced", "jaded", or even "cynical", but I know of too many instances of vaporware to be swayed by promises of software that hasn't yet been written.

I just call you cynical. We can't do anything if we don't try to do it.

 

 

- Easier integration with superclasses that expose their own parameters and use different modules to build up full applications stacks or complex setups that involve multiple modules.

Sorry, I'm not seeing it.

 

- A PuppetLabs and/or Community driven central repository of well tested and features rich unique Standard modules
(it might be a subset of the Forge or a set of git repos with only one module for application)

Yes, the idea of a curated repository of interoperable modules has been raised before.  I think it's a fine idea, but I don't think it depends on parameter naming conventions for realization.  In fact, I don't think parameter naming conventions actually bring much to that particular table. 

I think exactly the contrary.
Having a curated repository of interoperable modules without a minimum set of shared parameters would be a great drawback and a major lost occasion.
Not everybody has the skills and time to study a module's documentation or even internal code to understand the logic of its parameters, being able to leverage on some common parameters for the most important and common use cases (such as defining the configuration file to use) would help and save the time of many people, both beginners and experienced.

  

 

 

> None of that is a fundamental reason to object to the effort, but I'm not
> seeing any promise of significant benefit to motivate me to participate
> actively.
>
> I do have a bona fide objection, however: although the effort is cast at
> this point as being aimed exclusively at parameter naming, by implication it
> also covers elements of module structure, organization, and style as well,
> as the things being named have to exist for the standard to be relevant.  I
> do understand that the intention is not to make all the standardized
> controls mandatory for every module, but for those that a given module does
> provide, even the standard's limited reusability and interoperability goals
> are not served if those controls are not located where the standard
> anticipates (which is for the most part on a single front-end class). 

The assumption is that parameters are passed to the main module class or to single specific defines.
This is the only constraint to the module structure then you can organize the modules as you want. 

It is a non-trivial constraint that all of a module's external interface be implemented in the form of a single class and zero or more types (and zero or more custom functions, heretofore not discussed).  That it can be expressed in one sentence does not make it less significant.  Furthermore, there is an implicit assumption even in that admission: that module interfaces will involve parameterized classes.  That in itself is probably not a big deal any longer, but some of the prospective advantages claimed seem predicated on the premise that parameterized-style class declarations will be used.  That is a big deal, especially given that a large ecosystem of mutually interoperable modules is hypothesized as a viable and likely outcome.

A really reusable module HAS to be parametrized.
Take note, I can repeat and argument that anytime.
Parameters are the API of a module, the interface you can interact with it without touching it.
If you don't use parameters either you are forcing inside the module behaviours that can't be changed by users or you are forcing a specific way to obtain the data that affects the modules behaviour and this inherently makes it not reusable for people who provide data in other ways.

 

Most importantly, however, my central objection here is that a proposal that purports to be exclusively about naming parameters in fact has any implications at all for module organization and style.

Ok, it's not a proposal that purports to be exclusively about naming parameters: the usage of specific parameters involve patterns that enhance modules' reusability  and interoperability.
The sample module code definitively exposes design patterns with modules (and it demonstrates that you can follow different patterns to achieve the same behaviour.
 
Reading it again this is not so clear in the first post here, but it's widely expressed in the linked pages.
So, sorry, now let's move a step further, let's talk, if you want, about how to make that paper/draft/proposal better.
And if you think that such a thing should not even be discussed, well, ok, I'll accept that and discuss it with who is interested in it.
  

Fair point, its a difficult document to position precisely. I guess I
foresee this as something that should be a part of a guide for writing
modules then any hard/fast rule or 'standard'.

Call it position paper or draft for a proposed standard, what's important, imho, is the final goal:

The names of things matter, as indeed this whole discussion presupposes.  How people respond to the initiative will be colored on many levels by what you call it and how you position it.  In fact, I might not have joined the discussion if the effort had been positioned as targeting a set of common conventions, but setting a standard is a rather different endeavor.

 

some recommendations for naming conventions to use in modules (parameters) that can help in making some order in the modules' casbah while suggestion patterns that enhance reusability and may improve interoperabiity.

Since it's at least 3 years that there're some sparse discussions about modules reusability I preferred to give some solid ground upon which have a discussion, for this reason I placed the links of  the stdmod sample modules and the Google Doc.

Please consider this as a starting point to improve and eventually radically change (draft for 0.0.1 means that there's STILL to be a 0.0.1 and whatever comes after).

Thank you guys for the insights, hope to see your contributions to the Doc, both for the parts that find you skeptic and the parts you agree upon. I think it's important to do something that is well thought and shared but is even more important to start to do it, step by step, and I hope we all agree that is is something to do, sooner or later to avoid total chaos and fragmentation in the modules ecosystem.

I think you have embarked on a worthy effort, but I also think your aspirations are too grand.  Build it, with as much community participation as you can reasonably accommodate, and let the result speak for itself.  If it succeeds well in everything you envision then it is likely to be well received and broadly implemented.

To be honest I already did something similar for my own set of modules.
I am surely biased but having common variables definitively gives a sense to the term "least surprise".
About reusability I can say that I'm using those modules exactly as they are on GitHub in several totally different Puppet installations and often the implementation of a new service or role is as quick as writing few lines of predictable code.

Somehow it also seems that some other people are doing the same with them, given some figures from GitHub and the fact that about 100 different committers have improved or patched them.

One of the reasons why I'm basically considering to thrash all this "legacy" work and trying to promote and embrace wider standards is that I definitively think that they can give great benefits  to Puppet's modules ecosystem. 
I'm well aware that such or a similar effort can really succeed only if PuppetLabs endorses it, so I'd be ready to step aside and hand everything to more expert and authoritative hands.

Still this idea is not only mine, I've talked and shared it with various other Puppetteers (some of them started to make  modules even before me or you even knew what Puppet was) and the consensus is wide enough to make me think that even without an explicit support from PuppetLabs, a robust, coherent, shared and single repository  of Puppet modules made by (some) community members can be done and can work well.

But do not set out with the expectation of that the result will be adopted or used otherwise, and certainly do not create a chicken-and-egg problem, where the effort's success depends on it being successful.

 

Will it be successful? 
We'll never know if we don't start to work on it.

Al

[David Schmitt]

On 20.06.2013 00:34, Alessandro Franceschi wrote:
> Having a curated repository of interoperable modules without a minimum
> set of shared parameters would be a great drawback and a major lost
> occasion.

Yes.

> _A really reusable module HAS to be parametrized_.

Yes.

> Somehow it also seems that some other people are doing the same with
> them, given some figures from GitHub and the fact that about 100
> different committers have improved or patched them.

Yes.

> One of the reasons why I'm basically considering to thrash all this
> "legacy" work and trying to promote and embrace wider standards is that
> I definitively think that they can give great benefits to Puppet's
> modules ecosystem.

Yes.

> Still this idea is not only mine, I've talked and shared it with various
> other Puppetteers (some of them started to make modules even before me
> or you even knew what Puppet was) and the consensus is wide enough to
> make me think that even without an explicit support from PuppetLabs, a
> robust, coherent, shared and single repository of Puppet modules made
> by (some) community members can be done and can work well.

Yes.



Regards, David

[jcbollinger]

On Wednesday, June 19, 2013 5:34:58 PM UTC-5, Alessandro Franceschi wrote:

On Wednesday, June 19, 2013 5:03:41 PM UTC+2, jcbollinger wrote:

Call me "experienced", "jaded", or even "cynical", but I know of too many instances of vaporware to be swayed by promises of software that hasn't yet been written.

I just call you cynical. We can't do anything if we don't try to do it.

 

By all means, do.  If you are somehow getting the impression that I object to the project then I apologize.  Indeed, I encourage you to proceed, and I wish you well.  I just have low regard for software that doesn't yet exist.  I do not find the potential for nebulous future software a persuasive argument for adopting anything.

What I actually object to is misrepresentation -- even though accidental -- of the nature and scope of the project.  It is NOT restricted to choosing names.

Secondarily, I object to positioning the effort as a standardization project, and I advise you to position it differently.  Names and labels are meaningful.  How the project proceeds, who participates, what they propose, and all manner of intangibles will be influenced by your decisions in this regard.  Frankly, I am inclined to resist a "standardization" effort from any random third party, but I have no reason whatever to oppose anyone drafting a set of conventions or best practices.  After they have been tested in the field will be a suitable time to consider standardization or endorsement.

A really reusable module HAS to be parametrized.

Utter hogwash.

 

Take note, I can repeat and argument that anytime.

Then no doubt we'll be hearing from each other on this again.

 

Parameters are the API of a module, the interface you can interact with it without touching it.
If you don't use parameters either you are forcing inside the module behaviours that can't be changed by users or you are forcing a specific way to obtain the data that affects the modules behaviour and this inherently makes it not reusable for people who provide data in other ways.

The data consumed by a module and the labels by which they are identified are certainly parts of a module's API, but it is by no means necessary to express those or interact with them in the form of class parameters.  You cannot safely use parameterized-style class declarations (as opposed to class definitions) of any API class, especially in or with modules intended to be interoperable.  That makes parameterization largely moot as far as reusability goes.  If people want to parameterize their classes then I have no special reason to object.  I do object, however, to the premise that doing so usefully improves reusability or interoperability, and therefore I object to any proposal that expressly calls for classes to be parameterized.

As long as we're talking about standards, Hiera is Puppet's de facto standard interface to external data.  It is usually expected that proposed standards will build on other, existing standards, whether de facto or formal.

I think your proposal would be stronger if it truly did focus on the data that might need to be supported by modules and the names / labels / keys by which they are identified.  If you choose a namespace-based system then you can align it to be consistent with class parameterization and automated parameter binding, yet not dependent on those.  I know you want more, but you should consider making a separate effort of the rest.

 

Most importantly, however, my central objection here is that a proposal that purports to be exclusively about naming parameters in fact has any implications at all for module organization and style.

Ok, it's not a proposal that purports to be exclusively about naming parameters: the usage of specific parameters involve patterns that enhance modules' reusability  and interoperability.
The sample module code definitively exposes design patterns with modules (and it demonstrates that you can follow different patterns to achieve the same behaviour.
 
Reading it again this is not so clear in the first post here, but it's widely expressed in the linked pages.
So, sorry, now let's move a step further, let's talk, if you want, about how to make that paper/draft/proposal better.
And if you think that such a thing should not even be discussed, well, ok, I'll accept that and discuss it with who is interested in it.
  

I think it would be better if it actually were narrowed in scope to identifying patterns in the data that modules consume, and choosing naming conventions for those data.  Better, and also easier to reach consensus on.  Include some comments about namespacing, especially in conjunction with accessing data via Hiera.  These issues are separate from and more fundamental than class parameterization or module organization.  They will, however, influence anyone who chooses to write modules with parameterized API classes.


John

[Alessandro Franceschi]

On Thursday, June 20, 2013 4:12:46 PM UTC+2, jcbollinger wrote:

On Wednesday, June 19, 2013 5:34:58 PM UTC-5, Alessandro Franceschi wrote:

On Wednesday, June 19, 2013 5:03:41 PM UTC+2, jcbollinger wrote:

Call me "experienced", "jaded", or even "cynical", but I know of too many instances of vaporware to be swayed by promises of software that hasn't yet been written.

I just call you cynical. We can't do anything if we don't try to do it.

 

By all means, do.  If you are somehow getting the impression that I object to the project then I apologize.  Indeed, I encourage you to proceed, and I wish you well.  I just have low regard for software that doesn't yet exist.  I do not find the potential for nebulous future software a persuasive argument for adopting anything.

What I actually object to is misrepresentation -- even though accidental -- of the nature and scope of the project.  It is NOT restricted to choosing names.

Secondarily, I object to positioning the effort as a standardization project, and I advise you to position it differently.  Names and labels are meaningful.  How the project proceeds, who participates, what they propose, and all manner of intangibles will be influenced by your decisions in this regard.  Frankly, I am inclined to resist a "standardization" effort from any random third party, but I have no reason whatever to oppose anyone drafting a set of conventions or best practices.  After they have been tested in the field will be a suitable time to consider standardization or endorsement.

A really reusable module HAS to be parametrized.

Utter hogwash.

Lol. Show me a real example of a reusable module without parametrized classes that at least manages a typical package/service/configuration file scenario.

Give me proofs, not words that I have to search on a dictionary.

 

 

Take note, I can repeat and argument that anytime.

Then no doubt we'll be hearing from each other on this again.

 

Parameters are the API of a module, the interface you can interact with it without touching it.
If you don't use parameters either you are forcing inside the module behaviours that can't be changed by users or you are forcing a specific way to obtain the data that affects the modules behaviour and this inherently makes it not reusable for people who provide data in other ways.

The data consumed by a module and the labels by which they are identified are certainly parts of a module's API, but it is by no means necessary to express those or interact with them in the form of class parameters.  You cannot safely use parameterized-style class declarations (as opposed to class definitions) of any API class, especially in or with modules intended to be interoperable.  That makes parameterization largely moot as far as reusability goes.  If people want to parameterize their classes then I have no special reason to object.  I do object, however, to the premise that doing so usefully improves reusability or interoperability, and therefore I object to any proposal that expressly calls for classes to be parameterized.

As long as we're talking about standards, Hiera is Puppet's de facto standard interface to external data.  It is usually expected that proposed standards will build on other, existing standards, whether de facto or formal.

Sadly, Hiera is not the only de facto standard interface to Puppet's external data. Variables set via an ENC are another widely used "standard".
Also I'd say that Hiera is standard with Puppet 3, not with earlier versions.
So please show me a reusable module without parameters and without hiera functions inside, as that would not be reusable by whoever does not use Hiera.

I think your proposal would be stronger if it truly did focus on the data that might need to be supported by modules and the names / labels / keys by which they are identified.  If you choose a namespace-based system then you can align it to be consistent with class parameterization and automated parameter binding, yet not dependent on those.  I know you want more, but you should consider making a separate effort of the rest.

I've no problems in concentrating in naming without "forcing" the usage of parametrized classes, but still have to figure out HOW to manage injection of external data to the module without using parameters (and without using hiera calls or references to arbitrary fully qualified variables names).
Curiously one of my efforts in the past has been exactly towards this direction, so I have actually a solution for that, but it still requires a parametrized class, even if you can use it with a simple include.

 

Most importantly, however, my central objection here is that a proposal that purports to be exclusively about naming parameters in fact has any implications at all for module organization and style.

Ok, it's not a proposal that purports to be exclusively about naming parameters: the usage of specific parameters involve patterns that enhance modules' reusability  and interoperability.
The sample module code definitively exposes design patterns with modules (and it demonstrates that you can follow different patterns to achieve the same behaviour.
 
Reading it again this is not so clear in the first post here, but it's widely expressed in the linked pages.
So, sorry, now let's move a step further, let's talk, if you want, about how to make that paper/draft/proposal better.
And if you think that such a thing should not even be discussed, well, ok, I'll accept that and discuss it with who is interested in it.
  

I think it would be better if it actually were narrowed in scope to identifying patterns in the data that modules consume, and choosing naming conventions for those data.  Better, and also easier to reach consensus on.  Include some comments about namespacing, especially in conjunction with accessing data via Hiera.  These issues are separate from and more fundamental than class parameterization or module organization.  They will, however, influence anyone who chooses to write modules with parameterized API classes.

No problem in that. The intention is to share naming standards, not to force people in making modules in a certain way.

Still I keep on thinking that you need parametrized classes to have truly reusable modules (note that this does not mean automatically that you must use a class { 'name' : } statement to use them, on my modules you can set parameters how and where you want (hiera, enc/top scope) and just include classes that use these parameters.

Show me alternative code and I will gladly change my mind.

al

[David Schmitt]

On 2013-06-20 16:12, jcbollinger wrote:
> I just have low regard for software that
> doesn't yet exist. I do not find the potential for nebulous future
> software a persuasive argument for adopting anything.

You might want to check out the significant amount of modules available
at http://github.com/Example42 that are the source of these naming
conventions.

Regards, David

[jcbollinger]

I am aware of them, and of their connection to this discussion, but they are nothing to do with my quoted comment.  A check of the context will show that I was talking about this:

> but I see various other clear benefits, in the mid term:

> - The possibility to have an unified approach to smoke testing of common features 

> - The possibility to have web front-ends and ENC that leverage on the standardized parameters

John

[jcbollinger]

On Thursday, June 20, 2013 10:14:09 AM UTC-5, Alessandro Franceschi wrote:

On Thursday, June 20, 2013 4:12:46 PM UTC+2, jcbollinger wrote:

On Wednesday, June 19, 2013 5:34:58 PM UTC-5, Alessandro Franceschi wrote:

A really reusable module HAS to be parametrized.

Utter hogwash.

Lol. Show me a real example of a reusable module without parametrized classes that at least manages a typical package/service/configuration file scenario.

Give me proofs, not words that I have to search on a dictionary.

 

Anything you can configure with class parameters, you can configure without them via external data. Generally speaking, I would turn to hiera for that.  If you want an ENC that surpasses hiera's built-in capabilities, then it can be written in the form of a custom hiera back-end.

On the other hand, if your criterion for a module being "reusable" is that it can be used in every way exactly as a module based on parameterized classes can be used, then the question is rigged.

That hiera was not built-in to Puppet 2 does not alter my evaluation here.  It is readily available and widely used with that series.  I have always held that parameterized classes were not safe to use in Puppet 2 anyway, so if the objective is modules that are reusable in Puppet 2 then I would go beyond "parameterization is not required" to "parameterization of API classes is forbidden".  That leaves you with dynamic scoping and extlookup() as your only built-in alternatives, and however suitable they may be (or not) for Puppet 2, they are not suitable for Puppet 3.

If the objective is for modules to be written so as to be usable in both Puppet 2 and Puppet 3, then I think Hiera is the only viable alternative.  Classes belonging to module APIs must not be declared via the parameterized style, and though avoiding that in Puppet 2 could be supported by using hiera() calls for parameter default values, writing API classes that way produces (i) worse compilation performance in Puppet 3, and (ii) the opportunity for users to shoot themselves in the foot.


John

[Alessandro Franceschi]

On Friday, June 21, 2013 5:05:20 PM UTC+2, jcbollinger wrote:

On Thursday, June 20, 2013 10:14:09 AM UTC-5, Alessandro Franceschi wrote:

On Thursday, June 20, 2013 4:12:46 PM UTC+2, jcbollinger wrote:

On Wednesday, June 19, 2013 5:34:58 PM UTC-5, Alessandro Franceschi wrote:

A really reusable module HAS to be parametrized.

Utter hogwash.

Lol. Show me a real example of a reusable module without parametrized classes that at least manages a typical package/service/configuration file scenario.

Give me proofs, not words that I have to search on a dictionary.

 

Anything you can configure with class parameters, you can configure without them via external data. Generally speaking, I would turn to hiera for that.  If you want an ENC that surpasses hiera's built-in capabilities, then it can be written in the form of a custom hiera back-end.

On the other hand, if your criterion for a module being "reusable" is that it can be used in every way exactly as a module based on parameterized classes can be used, then the question is rigged.

That hiera was not built-in to Puppet 2 does not alter my evaluation here.  It is readily available and widely used with that series.  I have always held that parameterized classes were not safe to use in Puppet 2 anyway, so if the objective is modules that are reusable in Puppet 2 then I would go beyond "parameterization is not required" to "parameterization of API classes is forbidden".  That leaves you with dynamic scoping and extlookup() as your only built-in alternatives, and however suitable they may be (or not) for Puppet 2, they are not suitable for Puppet 

If the objective is for modules to be written so as to be usable in both Puppet 2 and Puppet 3, then I think Hiera is the only viable alternative.  Classes belonging to module APIs must not be declared via the parameterized style, and though avoiding that in Puppet 2 could be supported by using hiera() calls for parameter default values, writing API classes that way produces (i) worse compilation performance in Puppet 3, and (ii) the opportunity for users to shoot themselves in the foot.

Still words and no samples John.

I show you again this one, then.

https://github.com/example42/puppet-stdmod

it does what we want.

Automatic Hiera lookups if you have Puppet3, and I hope data bindings are here to stay.
or parametrized classes declarations, which are finally getting some respect also from ENCs.

You can use it how you want.

And if you don't like how the module is done, as I presume, you can do a different version, like this:
https://github.com/example42/puppet-stdmodalt
or whatever other better layout you may want to produce.

They are based on parametrized classes, incidentally.

I'm still not able to make a truly reusable module in other ways. 
(And still don't see them around).

But this is not the point.

Are the parameters / hiera variables suggested useful?
Can some of them be considered useful for the module's reusability?
Which ones are good enough to be considered obvious and a logic?

Which ones make sense enough to have a common name and be suggested as standard?

Which other ones can be suggested?

Now, 

really,
I would prefer to discuss about that
with you,
if you want
and most of all
eveybody 

thinks this might a good thing for the evolution of the modules' ecosystem in the next years. 

(Hello PuppetLabs?)

[Wolf Noble]

I'd like to say I really like the intent behind this.

Yes, a name is important. So is continuity. I believe the underlying goal is to provide a collection of reference material. Once the skeleton of this material is generally agreed upon, it can be used as guidelines to the community for how to implement a common set of functionality with performance, and the greatest quantity of interoperability in mind.

How is this a bad thing?

The demonstrated example shows plenty of well thought out functionality. How might we better implement these functional knobs and levers?

Lets go beyond the fact that the only reason we call knobs, switches, and levers what we do is that communally we agreed that this long-ish thing protruding from this surface over here should be called a lever, that this definitive state toggle be called a switch, and this round, grabbable twisty thing here should be called a knob. Names are important, but not just because they're names, they're icons of functionality.

Consistently referring to a lever as a lever provides more intuitive context, but doesn't change how it operates. Does it do the same thing if you call it a widget? Sure! Does it help with the goal of communally establishing conventions of grounding and interoperability? Not so much.

So, on the one hand, no, this conversation is not purely about naming conventions. On the other, I don't think that's the point to focus on.

I can absolutely see the value in a few sets of versioned "interoperability reference" modules, which provide one a good working model of how to provide specific bits of functionality in as flexible a manner as necessary.

There are several different good module paradigms out there. I feel that fragmentation of design and implementation hinders the community more than a little bit. It confuses it, at the very least.

I do not believe a 'one-size-fits-all' paradigm would do us all that much good here either. You'd likely always be one side or the other of too much scaffolding for simple modules, and not enough flexibility for complicated ones.

I believe this means a few levels of module complexity would need to be modeled. Then the transitions between those states would be more straightforward. Someone could expand the functionality of your module with greater ease and consistency.

I would love to see other examples which provide similar levels of implemented functionality.

(... (:(: and then the bikeshedding and performance holy wars can really get going :):) ...)

What other good ways are there to easily write unit tests to validate the different behaviors of a module without parameters?
Facts?
Limited scope/functionality subclasses?


This is a very exciting subject of conversation. I am thankful to everyone participating in it for their thoughts and contributions.

[Alessandro Franceschi]

On Saturday, June 22, 2013 11:47:13 AM UTC+2, Wolf Noble wrote:

I'd like to say I really like the intent behind this.

Yes, a name is important. So is continuity. I believe the underlying goal is to provide a collection of reference material. Once the skeleton of this material is generally agreed upon, it can be used as guidelines to the community for how to implement a common set of functionality with performance, and the greatest quantity of interoperability in mind.

How is this a bad thing?

The demonstrated example shows plenty of well thought out functionality. How might we better implement these functional knobs and levers?

Yes, that's the kind of things I'd like to discuss.

 

Lets go beyond the fact that the only reason we call knobs, switches, and levers what we do is that communally we agreed that this long-ish thing protruding from this surface over here should be called a lever, that this definitive state toggle be called a switch, and this round, grabbable twisty thing here should be called a knob. Names are important, but not just because they're names, they're icons of functionality.

Consistently referring to a lever as a lever provides more intuitive context, but doesn't change how it operates. Does it do the same thing if you call it a widget? Sure! Does it help with the goal of communally establishing conventions of grounding and interoperability? Not so much.

So, on the one hand, no, this conversation is not purely about naming conventions. On the other, I don't think that's the point to focus on.

I can absolutely see the value in a few sets of versioned "interoperability reference" modules, which provide one a good working model of how to provide specific bits of functionality in as flexible a manner as necessary.

There are several different good module paradigms out there. I feel that fragmentation of design and implementation hinders the community more than a little bit. It confuses it, at the very least.

I do not believe a 'one-size-fits-all' paradigm would do us all that much good here either. You'd likely always be one side or the other of too much scaffolding for simple modules, and not enough flexibility for complicated ones.

Agree.

 

I believe this means a few levels of module complexity would need to be modeled. Then the transitions between those states would be more straightforward. Someone could expand the functionality of your module with greater ease and consistency.

Totally agree.

 

I would love to see other examples which provide similar levels of implemented functionality.

Actually it would be interesting, and actually is something that is definitively planned, to explore, different sample modules for different kind of applications: for example Java/Rails apps, or more complex cases, where you need defines but maybe also types and providers, also to validate and test the naming and functional choices.

(... (:(: and then the bikeshedding and performance holy wars can really get going :):) ...)

What other good ways are there to easily write unit tests to validate the different behaviors of a module without parameters?
Facts?
Limited scope/functionality subclasses?

I still can't figure out how to make a reusable module without parameters and without explicit hiera calls inside.

I'd really be interested in seeing an example.

 

This is a very exciting subject of conversation. I am thankful to everyone participating in it for their thoughts and contributions.

I'm looking forward for your suggestions on:
https://docs.google.com/a/lab42.it/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA

al 

[jcbollinger]

On Friday, June 21, 2013 6:21:22 PM UTC-5, Alessandro Franceschi wrote:

On Friday, June 21, 2013 5:05:20 PM UTC+2, jcbollinger wrote:

Anything you can configure with class parameters, you can configure without them via external data. Generally speaking, I would turn to hiera for that.  If you want an ENC that surpasses hiera's built-in capabilities, then it can be written in the form of a custom hiera back-end.

On the other hand, if your criterion for a module being "reusable" is that it can be used in every way exactly as a module based on parameterized classes can be used, then the question is rigged.

That hiera was not built-in to Puppet 2 does not alter my evaluation here.  It is readily available and widely used with that series.  I have always held that parameterized classes were not safe to use in Puppet 2 anyway, so if the objective is modules that are reusable in Puppet 2 then I would go beyond "parameterization is not required" to "parameterization of API classes is forbidden".  That leaves you with dynamic scoping and extlookup() as your only built-in alternatives, and however suitable they may be (or not) for Puppet 2, they are not suitable for Puppet 
If the objective is for modules to be written so as to be usable in both Puppet 2 and Puppet 3, then I think Hiera is the only viable alternative.  Classes belonging to module APIs must not be declared via the parameterized style, and though avoiding that in Puppet 2 could be supported by using hiera() calls for parameter default values, writing API classes that way produces (i) worse compilation performance in Puppet 3, and (ii) the opportunity for users to shoot themselves in the foot.

Still words and no samples John.

None of my modules have any parameterized classes, but I am not at liberty to publish them.  I can derive an example from someone else's module -- maybe yours -- and I will do so if you're actually interested.  However, you have now said at least twice that that's not what you want to talk about, and I'm not inclined to do the work if you're not interested in the result.

Either way, whether such modules are common is altogether beside the (my) point, which is about whether conventions such as you wish to develop and promote should require modules' API classes to be parameterized.  I will oppose that aspect of any such effort.

The more fundamental idea of a common mapping of data and data types to names (i.e. a data "ontology"), on the other hand, is an outstanding idea.  If the discussion is at that level then it will certainly keep my interest, and it may attract my active participation.


John

[Alessandro Franceschi]

On Monday, June 24, 2013 3:57:55 PM UTC+2, jcbollinger wrote:

On Friday, June 21, 2013 6:21:22 PM UTC-5, Alessandro Franceschi wrote:

On Friday, June 21, 2013 5:05:20 PM UTC+2, jcbollinger wrote:

Anything you can configure with class parameters, you can configure without them via external data. Generally speaking, I would turn to hiera for that.  If you want an ENC that surpasses hiera's built-in capabilities, then it can be written in the form of a custom hiera back-end.

On the other hand, if your criterion for a module being "reusable" is that it can be used in every way exactly as a module based on parameterized classes can be used, then the question is rigged.

That hiera was not built-in to Puppet 2 does not alter my evaluation here.  It is readily available and widely used with that series.  I have always held that parameterized classes were not safe to use in Puppet 2 anyway, so if the objective is modules that are reusable in Puppet 2 then I would go beyond "parameterization is not required" to "parameterization of API classes is forbidden".  That leaves you with dynamic scoping and extlookup() as your only built-in alternatives, and however suitable they may be (or not) for Puppet 2, they are not suitable for Puppet 
If the objective is for modules to be written so as to be usable in both Puppet 2 and Puppet 3, then I think Hiera is the only viable alternative.  Classes belonging to module APIs must not be declared via the parameterized style, and though avoiding that in Puppet 2 could be supported by using hiera() calls for parameter default values, writing API classes that way produces (i) worse compilation performance in Puppet 3, and (ii) the opportunity for users to shoot themselves in the foot.

Still words and no samples John.

None of my modules have any parameterized classes, but I am not at liberty to publish them.  I can derive an example from someone else's module -- maybe yours -- and I will do so if you're actually interested.  However, you have now said at least twice that that's not what you want to talk about, and I'm not inclined to do the work if you're not interested in the result.

I'm still interested in seeing how this can be done, so any sample code useful to understand the approach would be extremely welcomed.

Just an example, then we can move on.

I'm not interested in sticking the discussion on that, also because I feel like we might have found the right "communication frequency"

 

Either way, whether such modules are common is altogether beside the (my) point, which is about whether conventions such as you wish to develop and promote should require modules' API classes to be parameterized.  I will oppose that aspect of any such effort.

The more fundamental idea of a common mapping of data and data types to names (i.e. a data "ontology"), on the other hand, is an outstanding idea.  If the discussion is at that level then it will certainly keep my interest, and it may attract my active participation.

Good.
Then consider that all the name for "parameters"  in:

https://docs.google.com/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA/edit?usp=sharing

may either refer to class' parameters or to hiera variables.
For example a parameter "ensure" on the module apache, would map to hiera('apache::ensure').

What of those parameters/hiera variables could be considered useful enough to be recommended?
What would be their names?

[jcbollinger]

On Monday, June 24, 2013 9:12:17 AM UTC-5, Alessandro Franceschi wrote:

On Monday, June 24, 2013 3:57:55 PM UTC+2, jcbollinger wrote:

None of my modules have any parameterized classes, but I am not at liberty to publish them.  I can derive an example from someone else's module -- maybe yours -- and I will do so if you're actually interested.  However, you have now said at least twice that that's not what you want to talk about, and I'm not inclined to do the work if you're not interested in the result.

I'm still interested in seeing how this can be done, so any sample code useful to understand the approach would be extremely welcomed.

Just an example, then we can move on.

So, here is a version of puppet-stdmod without class parameterization: https://github.com/jcbollinger/puppet-stdmod .

All management capabilities of the original module are retained.  Customization data are provided to it via Hiera (which the module calls explicitly), so the most significant change was simply to convert parameters to ordinary variables initialized vie Hiera.

The conversion was not entirely mechanical, in that I had to accommodate some Hiera idiosyncrasies -- mainly that it cannot return either undef or (boolean) false.  Inasmuch as one would like the parameterized version to be fully compatible with automatic parameter binding, however, I would account at least some of those adaptations as usability enhancements with respect to the parameterized version.

I did not convert the tests.

Also, I do not specially endorse stdmod's particular design (either version).  I would write it at least a bit differently if doing so from scratch, but I think we've agreed to turn the discussion in a different direction.


John

[Dan White]

I have been watching this thread grow and metastasize and I would like to offer my opinions:

"The wonderful thing about standards is that there are so many of them to choose from." Grace Hopper (maybe)

There are at least two very different directions to approach this from, IMHO.

The first is from the idea to create a universally useable module that can be dropped into any puppet master and run without modification. This seems to me to be the approach of Example-42.

The other approach is to provide a set of common sense, best-practice guidelines for folks to use to "grow" their own modules for their own specific needs.

I think both are necessary and valuable.

My grumble about the Example-42-Swiss-Army-Knife approach is that I would like to know what is absolutely necessary and what is optional. I do not want to implement the equivalent of the Sears Craftsman Gazillon Piece Tool Warehouse when all I need is one lousy Phillip's screwdriver.

I have dug thru some of the Example 42 modules and I have learned more than a few nifty ways of getting things done, but I am working in a one-operating-system environment (Red Hat) and I do not need all the extra code to cover the other OS's.

Other grumbles (not only about Example 42) include sparse documentation and examples: If you are going to define a pile of parameters, it would be very helpful to include examples to show how to use them. Very few published modules provide enough detail to make me happy.

The old saying is "There's more than one way to skin a cat." (My apologies to all ailurophiles out there, of which I am one)
Same thing here. Puppet is a potentially powerful framework for system configuration and management. If you are sloppy about how you use it, you can booger up your systems big-time and lose important data.

Exactly how to do it all is not as important (IMHO) as learning right and wrong ways to do individual tasks. It is then up to the individual DevOp to tailor the pieces to fit their particular needs and environment.

And to use the closing of Dennis Miller, "...of course, that's just my opinion . I could be wrong."

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

[Alessandro Franceschi]

On Tuesday, June 25, 2013 9:16:20 PM UTC+2, jcbollinger wrote:

On Monday, June 24, 2013 9:12:17 AM UTC-5, Alessandro Franceschi wrote:

On Monday, June 24, 2013 3:57:55 PM UTC+2, jcbollinger wrote:

None of my modules have any parameterized classes, but I am not at liberty to publish them.  I can derive an example from someone else's module -- maybe yours -- and I will do so if you're actually interested.  However, you have now said at least twice that that's not what you want to talk about, and I'm not inclined to do the work if you're not interested in the result.

I'm still interested in seeing how this can be done, so any sample code useful to understand the approach would be extremely welcomed.

Just an example, then we can move on.

So, here is a version of puppet-stdmod without class parameterization: https://github.com/jcbollinger/puppet-stdmod .

Ok, this is what I expected.
Thank you for your time.

I can definitively argue that this is not fully reusable

  ("So please show me a reusable module without parameters and without hiera functions inside, as that would not be reusable by whoever does not use Hiera.")
but it's not relevant since with Puppet 3 data bindings we have the best of both worlds.
Personally I see only good reasons to recommend, as patterns for the future, modules designed with parametrized classes.

All management capabilities of the original module are retained.  Customization data are provided to it via Hiera (which the module calls explicitly), so the most significant change was simply to convert parameters to ordinary variables initialized vie Hiera.

The conversion was not entirely mechanical, in that I had to accommodate some Hiera idiosyncrasies -- mainly that it cannot return either undef or (boolean) false.  Inasmuch as one would like the parameterized version to be fully compatible with automatic parameter binding, however, I would account at least some of those adaptations as usability enhancements with respect to the parameterized version.

I did not convert the tests.

Also, I do not specially endorse stdmod's particular design (either version).  I would write it at least a bit differently if doing so from scratch, but I think we've agreed to turn the discussion in a different direction.


John

Yes,

let's go on.
I guess the first steps should be to identify a coherent naming pattern, even before single names and define more precisely what parameters might be recommended, optional, or considered "extensions" . 

The google doc is always open to scribble some ideas and proposals.

Al

[Alessandro Franceschi]

On Tuesday, June 25, 2013 10:10:13 PM UTC+2, Ygor wrote:

I have been watching this thread grow and metastasize and I would like to offer my opinions:

"The wonderful thing about standards is that there are so many of them to choose from." Grace Hopper (maybe)

There are at least two very different directions to approach this from, IMHO.

The first is from the idea to create a universally useable module that can be dropped into any puppet master and run without modification. This seems to me to be the approach of Example-42.

The other approach is to provide a set of common sense, best-practice guidelines for folks to use to "grow" their own modules for their own specific needs.

I think both are necessary and valuable.

My grumble about the Example-42-Swiss-Army-Knife approach is that I would like to know what is absolutely necessary and what is optional. I do not want to implement the equivalent of the Sears Craftsman Gazillon Piece Tool Warehouse when all I need is one lousy Phillip's screwdriver.

Well the idea is that in the Fat Warehouse you pick the screwdriver you need, the others are there, at disposal, mostly harmless.

I have dug thru some of the Example 42 modules and I have learned more than a few nifty ways of getting things done, but I am working in a one-operating-system environment (Red Hat) and I do not need all the extra code to cover the other OS's.

Others will.
The first step for modules' reusability is support of multiple OS.

Note, IMHO a reusable module is not always the right choice:
slim and essential code for very specific and internally standardised setups can be preferred at times.

Other grumbles (not only about Example 42) include sparse documentation and examples: If you are going to define a pile of parameters, it would be very helpful to include examples to show how to use them. Very few published modules provide enough detail to make me happy.

That's another good reason to standardize the names of common sense parameters, that can therefore be better documented and explained. 

The old saying is "There's more than one way to skin a cat." (My apologies to all ailurophiles out there, of which I am one)
Same thing here. Puppet is a potentially powerful framework for system configuration and management. If you are sloppy about how you use it, you can booger up your systems big-time and lose important data.

Exactly how to do it all is not as important (IMHO) as learning right and wrong ways to do individual tasks. It is then up to the individual DevOp to tailor the pieces to fit their particular needs and environment.

Yes, but when this tailoring becomes "take existing modules and modify or write them from scratch " you end up seeing dozens of randomly documented and designed Mysql modules, that can be used in different ways and may have weird cross dependencies.

That's why starting for some naming standards helps.

That's also why I personally think that some common modules like apache, mysql, nginx, postgres or other that are frequently used by third modules should have explicit and dedicated standard parameters for the typical defines they provide.
 

my2c
al

[Alessandro Franceschi]

Talking about naming schemes, here are some possibilities (copied from: https://docs.google.com/a/lab42.it/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA ) we can discuss and improve.

Given the sample names below, there are at least  2 different approaches that can be followed, and once defined all the parameters names can somehow emerge naturally:
1- Place a common suffix for any kind of parameter
2- Place suffixes for some specific params and leave more general names of general ones

Examples for case 1 ensure package_name package_version service_name service_ensure file_path file_source file_template dir_path dir_source

Examples for case 2 ensure package version service service_ensure file_path source template dir_path dir_source

For both cases extra resources can be referred according to their name and type of resource, for example: server_package_name, master_file_path, controller_service_name and things like that.

A general pattern could be:
[name_of_resource]_<type_of_resource>_<resource_parameter>
- For the “main” resources of a class (main/unique package, main/unique service, main configuration file...) the name_of_resource is omitted.
- The type_of_resource reflects Puppet types (are “exceptions” like dir_ allowed?)
- The resource_parameter tends to map to a typical Puppet resource parameter (exceptions like template, version, options?) 

[jcbollinger]

On Tuesday, June 25, 2013 4:49:49 PM UTC-5, Alessandro Franceschi wrote:

On Tuesday, June 25, 2013 9:16:20 PM UTC+2, jcbollinger wrote:

So, here is a version of puppet-stdmod without class parameterization: https://github.com/jcbollinger/puppet-stdmod .

Ok, this is what I expected.

Of course it is.  I told you in advance what it was going to be.

 

Thank you for your time.

I can definitively argue that this is not fully reusable

  ("So please show me a reusable module without parameters and without hiera functions inside, as that would not be reusable by whoever does not use Hiera.")

And I rejected that standard of reusability, for sound reasons that you did not rebut.  Since we are closing off that line of discussion, we will have to agree to disagree.

Yes,

let's go on.
I guess the first steps should be to identify a coherent naming pattern, even before single names and define more precisely what parameters might be recommended, optional, or considered "extensions" . 

Before any decision is made on naming patterns -- or at least in conjunction with that -- don't you think we should come to grips with the patterns of data to which we hope to assign names?  The Google doc does address that topic, at least implicitly, but I think we should be satisfied with the coverage and characterization of the data, else we cannot be confident that the chosen names are the most appropriate ones.  I will devote some attention to that, but I hope I will not be the only one to do so.


John

[Alessandro Franceschi]

On Wednesday, June 26, 2013 3:37:10 PM UTC+2, jcbollinger wrote:

On Tuesday, June 25, 2013 4:49:49 PM UTC-5, Alessandro Franceschi wrote:

On Tuesday, June 25, 2013 9:16:20 PM UTC+2, jcbollinger wrote:

So, here is a version of puppet-stdmod without class parameterization: https://github.com/jcbollinger/puppet-stdmod .

Ok, this is what I expected.

Of course it is.  I told you in advance what it was going to be.

 

Thank you for your time.

I can definitively argue that this is not fully reusable

  ("So please show me a reusable module without parameters and without hiera functions inside, as that would not be reusable by whoever does not use Hiera.")

And I rejected that standard of reusability, for sound reasons that you did not rebut.  Since we are closing off that line of discussion, we will have to agree to disagree.

Actually I DID rebut and anticipated them. That module would not work on at least ten different Puppet setups where I've worked and Hiera was not used.

But, promise, I won't come back on this, let's happily disagree :-) 

Yes,

let's go on.
I guess the first steps should be to identify a coherent naming pattern, even before single names and define more precisely what parameters might be recommended, optional, or considered "extensions" . 

Before any decision is made on naming patterns -- or at least in conjunction with that -- don't you think we should come to grips with the patterns of data to which we hope to assign names?  The Google doc does address that topic, at least implicitly, but I think we should be satisfied with the coverage and characterization of the data, else we cannot be confident that the chosen names are the most appropriate ones.  I will devote some attention to that, but I hope I will not be the only one to do so.

Good point.
In the doc I placed some possible examples on where to assign names, but more (or remarks on the ones listed) cases are welcomed.
(Let me have your email for an invitation to edit)

Btw, comments and access to:
https://docs.google.com/a/lab42.it/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA
is open to everybody, and whoever writes modules should be somehow concerned.

At the moment the discussion seems limited to very few people (even if I know various others are silently lurking :-)

The same place of the document is to be considered temporary.

Al

[Ryan Coleman]

Hi Al, et al. I apologize for being so late to this party but I'm loving all the conversation around this. I've read through the Google Doc and found lots of cool ideas on class parameter names. Please forgive me though, my product manager wired brain wants to pause at this point and discuss it a bit first before I offer my opinions on the doc. 

On Tue, Jun 18, 2013 at 11:19 AM, Alessandro Franceschi <a...@lab42.it> wrote:

For me a module is reusable when:
1- It supports many different OS (this is somehow implicit and does not involve naming conventions)

2- It leaves to the user freedom on how to populate and customize the provided configuration files ( I think this is the main point for most reusability cases)
3- It allows  the user to manage some behaviours of the module (has a service to be restarted after a file change? Do I want to manage a service status (at runtime or boot)
4- In (somehow extreme) cases it allows the user to customize names of package/services, paths of files and so on

5- It allows seamless addition of custom resources, not managed by the module but related to it

6- It allows the user to decide where to place his data (also this is out of naming convention scope)

I'll admit that this is my Forge biased view of things, but I'm working towards modules that are reusable, interoperable and introspectable. It would help me contribute to the discussion if we could hammer out whether we loosely agree on the goal and definitions. I'm already pretty happy with your definition of reusable, but I'll paraphrase.

Interoperable

- Module A is known to do X, Y & Z.
- Module B also does X, Y & Z and can seamlessly replace module A


Reusable

- Supports multiple operating systems with a clear & standard pattern for adding additional OSes
- General capabilities of module can be switched on or off or lightly modified. Ex., don't manage services or override configuration directory.

One way that we differ immediately on reusability is that you're pretty detailed on what you should be able to customize, like package and service names. I don't disagree with you but I'm trying to start from a higher level and see whether that's sufficient. I'm not sure what the balance is regarding # of class parameters in use / ease of use.


Introspectable

- Code follows style guide and other patterns so that contributions are more easily made and managed.
- Puppet should be able to programmatically tell us about defined class parameters and their default values. (yeah, this is theoretical atm)

Are these three goals and their values what we're all striving for with this proposal? If not, what am I missing or getting wrong? 

Thanks for kicking this off Al. I also care deeply about this stuff and will be trying to carve off more time each week to help you continue to explore it.

--
Ryan Coleman | Modules & Forge | @ryanycoleman | ryancoleman in #puppet

 

[Alessandro Franceschi]

On Monday, July 1, 2013 5:21:44 AM UTC+2, Ryan Coleman wrote:

Hi Al, et al. I apologize for being so late to this party but I'm loving all the conversation around this. I've read through the Google Doc and found lots of cool ideas on class parameter names. Please forgive me though, my product manager wired brain wants to pause at this point and discuss it a bit first before I offer my opinions on the doc. 

On Tue, Jun 18, 2013 at 11:19 AM, Alessandro Franceschi <a...@lab42.it> wrote:

For me a module is reusable when:
1- It supports many different OS (this is somehow implicit and does not involve naming conventions)

2- It leaves to the user freedom on how to populate and customize the provided configuration files ( I think this is the main point for most reusability cases)
3- It allows  the user to manage some behaviours of the module (has a service to be restarted after a file change? Do I want to manage a service status (at runtime or boot)
4- In (somehow extreme) cases it allows the user to customize names of package/services, paths of files and so on

5- It allows seamless addition of custom resources, not managed by the module but related to it

6- It allows the user to decide where to place his data (also this is out of naming convention scope)

I'll admit that this is my Forge biased view of things, but I'm working towards modules that are reusable, interoperable and introspectable. It would help me contribute to the discussion if we could hammer out whether we loosely agree on the goal and definitions. I'm already pretty happy with your definition of reusable, but I'll paraphrase.

Interoperable

- Module A is known to do X, Y & Z.
- Module B also does X, Y & Z and can seamlessly replace module A

Agree, and to clarify this even more:
If module C needs X which is provided by module A and B:
- A and B should be seamless replaceable (as you said)
- C should allow easy selection (for the user) of another module that provides X 

This second point might look redundant, given the first one, but it might be necessary since seamless  replacement of modules won't be easy to achieve OR a user may want to use a module that does not support the standard namings.

An approach for this second point is the usage of a dependency_class parameter (details in the Google Doc), but I'm sure the collective wisdom here may find a better solution.

Also this interoperability should be somehow managed at Modulefile level (and with the puppet module command), so that I can use B even if in the Modulefile is required A.
This part, which involves changes in Puppet's code, has to be somehow addressed sooner or later, IMHO.

Reusable

- Supports multiple operating systems with a clear & standard pattern for adding additional OSes

- General capabilities of module can be switched on or off or lightly modified. Ex., don't manage services or override configuration directory.

One way that we differ immediately on reusability is that you're pretty detailed on what you should be able to customize, like package and service names. I don't disagree with you but I'm trying to start from a higher level and see whether that's sufficient. I'm not sure what the balance is regarding # of class parameters in use / ease of use.

The idea is that you are not forced to provide all the "proposed" parameters, but if some of the parameters you provide in your module have the same function of the ones proposed, you should call them in the "standard" way.

We might classify parameters in different ways, so that some should be considered somehow recommended (for a "standard module"), other optional, and other  "extra" or "enhanced", because more specific and exotic (thinking about the ones related to monitoring/firewalling, for example).

 

Introspectable

- Code follows style guide and other patterns so that contributions are more easily made and managed.
- Puppet should be able to programmatically tell us about defined class parameters and their default values. (yeah, this is theoretical atm)

+1, but with a small note:

Even if I think it can be useful to provide some recommended patterns for modules design, this is a somehow more controversial and debatable matter (as the discussion with John in this thread may suggest) so I would not couple it strictly with the discussion about naming standards which it's, imho, more easy to stage and manage.
No problems for me in facing all the different problematics behind modules standards, but I'd suggest to make small steps and begin with the easier ones, since putting too many topics in the cauldron and keeping them tied together might make harder any improvement.
 

Are these three goals and their values what we're all striving for with this proposal? If not, what am I missing or getting wrong? 

I think we agree on the overall goals.

Actually you've added the one about introspectability that is a welcomed addition but it partly relies upon features that are still missing in Puppet.
My starting approach was to concentrate on things that can be done now, with the current Puppet versions (I'd leverage on Puppet 3, though, since we are talking about the future modules) and giving common names to common sense parameters seemed a minimal logical step.

Nevertheless, it's important to conduct this discussion in pair with the expected evolutions of Puppet, so that we can define something that makes sense now and will make sense in the future.

  

Thanks for kicking this off Al. I also care deeply about this stuff and will be trying to carve off more time each week to help you continue to explore it.

Thanks to you Ryan for supporting this, PuppetLabs' influence on the matter is vital.

Al 

[Alessandro Franceschi]

An update on the works.

There's a github organization where discussion and code can be provided in a more "neutral" way:
https://github.com/stdmod

The contents of the Good docs have been collected and moved to 

https://github.com/stdmod/puppet-modules

Here you can see a list of possible naming standards for different cases:

https://github.com/stdmod/puppet-modules/blob/master/Parameters_List.md

Incidentally I'd like to add / include there this suggestion from John (on another thread) as it makes sense and doesn't forse specific modules structures:
<< For example, in a class ntp::service, I would certainly choose the parameter name "ensure" over "service_ensure" >>

Some sample POC modules have been made in order to test and see how these naming convention actually work:

https://github.com/stdmod/puppet-openssh

https://github.com/stdmod/puppet-elasticsearch

the latter, in particular, even if not complete, I think is a good example of the reusability and interoperability options that these naming standard suggest.

al

On Monday, June 17, 2013 11:37:50 AM UTC+2, Alessandro Franceschi wrote:

Dear all,

I'd like to submit to the list's attention this post about Puppet Modules Standard naming conventions:
http://www.example42.com/?q=The_handy_Grail_of_Modules_Standards 

and  this working draft of a proposal for a version 0.0.1 of them:

https://docs.google.com/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA/edit?usp=sharing

The topic is not new, and on this list there have already been various discussions on how to manage modules' interoperability ( for example this interesting one on cross-modules dependencies: https://groups.google.com/forum/?fromgroups#!searchin/puppet-users/modules$20standards|sort:date/puppet-users/Fvl0aOe4RPE) .

Along with the current discussions on the topic and the Puppet evolution, I think that few simple *suggested* naming conventions on common parameters for classes and defines would benefit a lot the module's ecosystem with a small effort.

I hope the discussion will have a real follow up, this time, and possibly move under the more authoritative PuppetLabs umbrella, for the moment is just a personal proposal, even if shared, at least in the intentions, with many modules' authors and Puppetteers.

You can actively contribute to the proposed naming standards on the above Google document (let me know your gmail account) and you are invited to comment here what do you think of this (not new) idea: if you think that starting from simple naming convention is a step to be done, what's your general idea on the proposed naming standards, if you have better ideas on naming and approach. 

Let's grab this handy Grail :-)

Alessandro Franceschi

Example42

[Ryan Coleman]

Hey Al, sorry. I went on holiday and then had to unbury myself. I guess it's time I get my opinion out there. 

On Mon, Jul 1, 2013 at 2:33 AM, Alessandro Franceschi <a...@lab42.it> wrote:

On Monday, July 1, 2013 5:21:44 AM UTC+2, Ryan Coleman wrote:

Hi Al, et al. I apologize for being so late to this party but I'm loving all the conversation around this. I've read through the Google Doc and found lots of cool ideas on class parameter names. Please forgive me though, my product manager wired brain wants to pause at this point and discuss it a bit first before I offer my opinions on the doc. 

On Tue, Jun 18, 2013 at 11:19 AM, Alessandro Franceschi <a...@lab42.it> wrote:

For me a module is reusable when:
1- It supports many different OS (this is somehow implicit and does not involve naming conventions)

2- It leaves to the user freedom on how to populate and customize the provided configuration files ( I think this is the main point for most reusability cases)
3- It allows  the user to manage some behaviours of the module (has a service to be restarted after a file change? Do I want to manage a service status (at runtime or boot)
4- In (somehow extreme) cases it allows the user to customize names of package/services, paths of files and so on

5- It allows seamless addition of custom resources, not managed by the module but related to it

6- It allows the user to decide where to place his data (also this is out of naming convention scope)

I'll admit that this is my Forge biased view of things, but I'm working towards modules that are reusable, interoperable and introspectable. It would help me contribute to the discussion if we could hammer out whether we loosely agree on the goal and definitions. I'm already pretty happy with your definition of reusable, but I'll paraphrase.

Interoperable

- Module A is known to do X, Y & Z.
- Module B also does X, Y & Z and can seamlessly replace module A

Agree, and to clarify this even more:
If module C needs X which is provided by module A and B:
- A and B should be seamless replaceable (as you said)
- C should allow easy selection (for the user) of another module that provides X 

This second point might look redundant, given the first one, but it might be necessary since seamless  replacement of modules won't be easy to achieve OR a user may want to use a module that does not support the standard namings.

An approach for this second point is the usage of a dependency_class parameter (details in the Google Doc), but I'm sure the collective wisdom here may find a better solution.

As in, this is the class you're expected to depend on? That's going to be a little hard to standardize on. Sometimes it's a resource you want, sometimes a fact, maybe it's multiple classes.

 

Also this interoperability should be somehow managed at Modulefile level (and with the puppet module command), so that I can use B even if in the Modulefile is required A.
This part, which involves changes in Puppet's code, has to be somehow addressed sooner or later, IMHO.

I couldn't agree with you more here. We've got to allow for dependencies to be interchangeable and I think this is most elegantly solved (for the moment) at the module level, with its metadata and Forge ecosystem. That said, I think that's best left to another thread.

 

Reusable

- Supports multiple operating systems with a clear & standard pattern for adding additional OSes

- General capabilities of module can be switched on or off or lightly modified. Ex., don't manage services or override configuration directory.

One way that we differ immediately on reusability is that you're pretty detailed on what you should be able to customize, like package and service names. I don't disagree with you but I'm trying to start from a higher level and see whether that's sufficient. I'm not sure what the balance is regarding # of class parameters in use / ease of use.

The idea is that you are not forced to provide all the "proposed" parameters, but if some of the parameters you provide in your module have the same function of the ones proposed, you should call them in the "standard" way.

We might classify parameters in different ways, so that some should be considered somehow recommended (for a "standard module"), other optional, and other  "extra" or "enhanced", because more specific and exotic (thinking about the ones related to monitoring/firewalling, for example).

OK. I could get on board with that though I do think that the overall set of parameters needs to be pruned down. Which parameters could be better addressed with module/puppet changes? The dependency_class parameter mentioned above seems like a good example.

 

 

Introspectable

- Code follows style guide and other patterns so that contributions are more easily made and managed.
- Puppet should be able to programmatically tell us about defined class parameters and their default values. (yeah, this is theoretical atm)

+1, but with a small note:

Even if I think it can be useful to provide some recommended patterns for modules design, this is a somehow more controversial and debatable matter (as the discussion with John in this thread may suggest) so I would not couple it strictly with the discussion about naming standards which it's, imho, more easy to stage and manage.

I don't follow. Class parameters and their default values is controversial?

 

No problems for me in facing all the different problematics behind modules standards, but I'd suggest to make small steps and begin with the easier ones, since putting too many topics in the cauldron and keeping them tied together might make harder any improvement.

I respect that, but I think you'll agree that we may need big changes to solve some of these problems. I'm in a position to help with that, so let's strive for it!

 

 

Are these three goals and their values what we're all striving for with this proposal? If not, what am I missing or getting wrong? 

I think we agree on the overall goals.

Actually you've added the one about introspectability that is a welcomed addition but it partly relies upon features that are still missing in Puppet.
My starting approach was to concentrate on things that can be done now, with the current Puppet versions (I'd leverage on Puppet 3, though, since we are talking about the future modules) and giving common names to common sense parameters seemed a minimal logical step.

Nevertheless, it's important to conduct this discussion in pair with the expected evolutions of Puppet, so that we can define something that makes sense now and will make sense in the future.

I don't mean to beat this point to death, but I'm more interested in exploring what changes are needed to really solve the reusability goals we've discussed. I do think there's some value to having standard class parameters for module authors to align themselves around but it's not going to solve the problem IMO, it's just going to make it a little easier.

That said, I would like to simultaneously collaborate on the parameter standards, with an eye towards consolidation. Would you like that feedback in your Google Doc or the new GitHub repo?

 

  

Thanks for kicking this off Al. I also care deeply about this stuff and will be trying to carve off more time each week to help you continue to explore it.

Thanks to you Ryan for supporting this, PuppetLabs' influence on the matter is vital.

Yeah, no problem -- this is an awesome problem to solve. I am sorry that our involvement is not as active as you'd like. We're pushing really hard over here to clear the Forge and Puppet Labs modules backlog. I can only thank you for your passion and ask for your continued patience.

 

Al 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To post to this group, send email to puppet...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Ryan Coleman | Modules & Forge | ryanycoleman on twitter & #puppet IRC

 

[jcbollinger]

On Sunday, June 30, 2013 10:21:44 PM UTC-5, Ryan Coleman wrote:

I'll admit that this is my Forge biased view of things, but I'm working towards modules that are reusable, interoperable and introspectable. It would help me contribute to the discussion if we could hammer out whether we loosely agree on the goal and definitions. I'm already pretty happy with your definition of reusable, but I'll paraphrase.

Interoperable

- Module A is known to do X, Y & Z.
- Module B also does X, Y & Z and can seamlessly replace module A

That sounds eminently reasonable at the high level, but I'm not sure what it really means for module implementations.  I suspect that what you and Al are both getting at is that module interfaces should provide a consistent higher-level language built on top of Puppet DSL.  Certainly choosing parameter names in a consistent way advances that goal.  I see potential issues there, however, with class names, which conceivably differ from one module to another, whether simply by the choice of the author or as a result of differing module structure.

For example, consider two modules managing Puppet, one that provides a single front-end class by which agent and master can both be managed, and one providing separate classes for managing those services.  Although it might not be exactly difficult to switch from one to the other, doing so surely would involve more than just pulling one out of the module path and putting the other in.  It would involve more even than also universally changing the name of one or two referenced classes.  Would these two hypothetical modules therefore fail to be "interoperable" as you see it?

Perhaps you mean something different by "seamlessly".  To me, interoperability means largely that modules managing different subsystems can be deployed and used alongside one another without interfering with each other, either in terms of managed resource collisions or in terms of requirements.  If one can replace one module with another without creating these sorts of problems then that could be construed as "seamless", but it seems a little tangential to the line of discussion (and one that we've had before).

I still think the Puppet would be very well served by a facility such as we discussed here: https://groups.google.com/forum/#!topic/puppet-users/Fvl0aOe4RPE (naturally, I prefer my own proposal in that thread).  Were something like that available, I think it would have a significant impact on best practices for module implementation, and some impact on best practices for module interfaces.  Inasmuch as it needn't change any existing Puppet behavior, it could even be added to the Puppet 3 series.

 

Reusable

- Supports multiple operating systems with a clear & standard pattern for adding additional OSes
- General capabilities of module can be switched on or off or lightly modified. Ex., don't manage services or override configuration directory.

One way that we differ immediately on reusability is that you're pretty detailed on what you should be able to customize, like package and service names. I don't disagree with you but I'm trying to start from a higher level and see whether that's sufficient. I'm not sure what the balance is regarding # of class parameters in use / ease of use.

I'm with you as far as that goes, and I think starting with high-level principles is an advantageous approach.

I do think the reusability question to which Al and I devoted so much rhetoric rises to this level, or at least near it -- specifically, whether the "reusability" objective demands that module interfaces be implemented in terms of parameterized classes.  I appreciate that you may not want to come down on one side or the other at this point, and I'm not asking you to do, but I do observe that nothing among the criteria you so far advanced for this principle demand parameterization.  I suppose Al's point (6) addresses the question for him.  He and I differ on whether that's an appropriate criterion for reusability (regardless of whether it is a good or bad idea for some other reason), so that's something that you will eventually need to settle for yourself.

 

Introspectable

- Code follows style guide and other patterns so that contributions are more easily made and managed.
- Puppet should be able to programmatically tell us about defined class parameters and their default values. (yeah, this is theoretical atm)

Even if you suppose that classes will generally be parameterized, wouldn't it be reasonable to want to introspect explicit hiera() calls as well?  Even parameterized classes may in some cases perform them.

 

Are these three goals and their values what we're all striving for with this proposal? If not, what am I missing or getting wrong? 

I think they are all eminently worthy goals, though among them they considerably expand the scope of the current discussion as I previously perceived it.  I do not doubt that Al had broader ideas than the ones with which he started this thread, however.  Like Al, I would not necessarily have come up with introspection as a goal -- and I'm not sure it rises to the same level of abstraction as the others -- but I don't have any objection to it at this point.


John

[Alessandro Franceschi]

Hi Ryan

Personal and alternative time priorities and availability surely influence the flow of discussion and I definitively have to understand that better in the cases when I've some free time and others not, so sorry everybody for the insistence, and consider the pings about the matter as a way to keep the discussion alive.

On Saturday, July 13, 2013 6:04:34 PM UTC+2, Ryan Coleman wrote:

Hey Al, sorry. I went on holiday and then had to unbury myself. I guess it's time I get my opinion out there. 

On Mon, Jul 1, 2013 at 2:33 AM, Alessandro Franceschi <a...@lab42.it> wrote:

On Monday, July 1, 2013 5:21:44 AM UTC+2, Ryan Coleman wrote:

Hi Al, et al. I apologize for being so late to this party but I'm loving all the conversation around this. I've read through the Google Doc and found lots of cool ideas on class parameter names. Please forgive me though, my product manager wired brain wants to pause at this point and discuss it a bit first before I offer my opinions on the doc. 

On Tue, Jun 18, 2013 at 11:19 AM, Alessandro Franceschi <a...@lab42.it> wrote:

For me a module is reusable when:
1- It supports many different OS (this is somehow implicit and does not involve naming conventions)

2- It leaves to the user freedom on how to populate and customize the provided configuration files ( I think this is the main point for most reusability cases)
3- It allows  the user to manage some behaviours of the module (has a service to be restarted after a file change? Do I want to manage a service status (at runtime or boot)
4- In (somehow extreme) cases it allows the user to customize names of package/services, paths of files and so on

5- It allows seamless addition of custom resources, not managed by the module but related to it

6- It allows the user to decide where to place his data (also this is out of naming convention scope)

I'll admit that this is my Forge biased view of things, but I'm working towards modules that are reusable, interoperable and introspectable. It would help me contribute to the discussion if we could hammer out whether we loosely agree on the goal and definitions. I'm already pretty happy with your definition of reusable, but I'll paraphrase.

Interoperable

- Module A is known to do X, Y & Z.
- Module B also does X, Y & Z and can seamlessly replace module A

Agree, and to clarify this even more:
If module C needs X which is provided by module A and B:
- A and B should be seamless replaceable (as you said)
- C should allow easy selection (for the user) of another module that provides X 

This second point might look redundant, given the first one, but it might be necessary since seamless  replacement of modules won't be easy to achieve OR a user may want to use a module that does not support the standard namings.

An approach for this second point is the usage of a dependency_class parameter (details in the Google Doc), but I'm sure the collective wisdom here may find a better solution.

As in, this is the class you're expected to depend on? That's going to be a little hard to standardize on. Sometimes it's a resource you want, sometimes a fact, maybe it's multiple classes.

Well, whatever you need from an external module, in this class (for which the module author provides his default) can be replaced by another class, defined and provided by the users, which is supposed to provide the same resources (or declare classes and defines from other modules ).

A (not particular elegant) example is this:
https://github.com/stdmod/puppet-elasticsearch/blob/master/manifests/dependency.pp

which uses legacy resources (git::reposync from a example42 git module you may not use) and creates some opinionated resources like the init script for the elasticsearch service.

Another example is this:
https://github.com/example42/puppet-graylog2/blob/master/manifests/dependencies.pp
where is used a define like mongodb::user which may have different parameters or name or usage patterns in an alternative mongo module actually used by the graylog2 module user.
The same would apply for more common cases where in your module you have to use defines apache::vhost or mysql::grant (at least until we don't define a naming standard also for them specifically, but that's another point), if they can stay in a "replaceable" class the module is more reusable and more easily made cross-compatible.

I know this is a somehow partial and naif solution, but it's possible with the current language, and I agree that some higher level solutions  that may involve code changes in Puppet can and should be seeked. But IMHO one thing doesn't exclude the other.

Having some standard parameters names to refer to the name of classes provided by the module than can be replaced directly by the user, (dependency but maybe also other internally subclasses like user, install or even service and config...) for me it makes sense.

 

 

Also this interoperability should be somehow managed at Modulefile level (and with the puppet module command), so that I can use B even if in the Modulefile is required A.
This part, which involves changes in Puppet's code, has to be somehow addressed sooner or later, IMHO.

I couldn't agree with you more here. We've got to allow for dependencies to be interchangeable and I think this is most elegantly solved (for the moment) at the module level, with its metadata and Forge ecosystem. That said, I think that's best left to another thread.

+1

 

 

Reusable

- Supports multiple operating systems with a clear & standard pattern for adding additional OSes

- General capabilities of module can be switched on or off or lightly modified. Ex., don't manage services or override configuration directory.

One way that we differ immediately on reusability is that you're pretty detailed on what you should be able to customize, like package and service names. I don't disagree with you but I'm trying to start from a higher level and see whether that's sufficient. I'm not sure what the balance is regarding # of class parameters in use / ease of use.

The idea is that you are not forced to provide all the "proposed" parameters, but if some of the parameters you provide in your module have the same function of the ones proposed, you should call them in the "standard" way.

We might classify parameters in different ways, so that some should be considered somehow recommended (for a "standard module"), other optional, and other  "extra" or "enhanced", because more specific and exotic (thinking about the ones related to monitoring/firewalling, for example).

OK. I could get on board with that though I do think that the overall set of parameters needs to be pruned down. Which parameters could be better addressed with module/puppet changes? The dependency_class parameter mentioned above seems like a good example.

I'd say that the list of overall parameters to be considered a sort of "minimal standard requirement" is definitively to prune down, but I don't see any harm in defining standard name patterns for other parameters that are surely not requested by a standard module but may stay there, because the module author considered it worth the effort.

But, if there are some proposed parameters names that are considered not only useless but possibly harmful (because they imply a "wrong" design or might create confusion or whatever), then I definitively agree to amend them. 
For the dependency_class example, for example,  I think it's not harmful, at least given the current status of Puppet, but if everybody thinks the opposite... well I won't insist on that and follow the proposed alternative.

 

 

 

Introspectable

- Code follows style guide and other patterns so that contributions are more easily made and managed.
- Puppet should be able to programmatically tell us about defined class parameters and their default values. (yeah, this is theoretical atm)

+1, but with a small note:

Even if I think it can be useful to provide some recommended patterns for modules design, this is a somehow more controversial and debatable matter (as the discussion with John in this thread may suggest) so I would not couple it strictly with the discussion about naming standards which it's, imho, more easy to stage and manage.

I don't follow. Class parameters and their default values is controversial?

No,

but how to structure a module yes. 

For example you may have accept parameters only on the openssh main class, which may or may not include other subclasses or have separated openssh::client , openssh::server (but also openssh::service, params, config) subclasses  that can declared directly,  each with their own set of parameters.

The naming proposal should define patterns for both the cases, without requiring or involving a specific module design.

 

 

No problems for me in facing all the different problematics behind modules standards, but I'd suggest to make small steps and begin with the easier ones, since putting too many topics in the cauldron and keeping them tied together might make harder any improvement.

I respect that, but I think you'll agree that we may need big changes to solve some of these problems. I'm in a position to help with that, so let's strive for it!

Ok. 

I think we have somehow to narrow down and focus on the different naming proposals and decide which ones are reasonable enough to be considered somehow "obvious" and which ones require a larger and wider angled approach.

 

 

 

Are these three goals and their values what we're all striving for with this proposal? If not, what am I missing or getting wrong? 

I think we agree on the overall goals.

Actually you've added the one about introspectability that is a welcomed addition but it partly relies upon features that are still missing in Puppet.
My starting approach was to concentrate on things that can be done now, with the current Puppet versions (I'd leverage on Puppet 3, though, since we are talking about the future modules) and giving common names to common sense parameters seemed a minimal logical step.

Nevertheless, it's important to conduct this discussion in pair with the expected evolutions of Puppet, so that we can define something that makes sense now and will make sense in the future.

I don't mean to beat this point to death, but I'm more interested in exploring what changes are needed to really solve the reusability goals we've discussed. I do think there's some value to having standard class parameters for module authors to align themselves around but it's not going to solve the problem IMO, it's just going to make it a little easier.

That said, I would like to simultaneously collaborate on the parameter standards, with an eye towards consolidation. Would you like that feedback in your Google Doc or the new GitHub repo?

I'd say https://github.com/stdmod can be a good shared place where everybody can post and discuss code samples, naming proposals and implementations.
I'm open to allow any member / admin request to anybody.

And this Google group is the place where higher level discussions have their natural agora.

 

 

  

Thanks for kicking this off Al. I also care deeply about this stuff and will be trying to carve off more time each week to help you continue to explore it.

Thanks to you Ryan for supporting this, PuppetLabs' influence on the matter is vital.

Yeah, no problem -- this is an awesome problem to solve. I am sorry that our involvement is not as active as you'd like. We're pushing really hard over here to clear the Forge and Puppet Labs modules backlog. I can only thank you for your passion and ask for your continued patience.

I think and hope good things can come out about this whole matter at the PuppetConf, maybe in the developer day: it could be a great occasion to gather into live action who is interested in the matter and if we manage, as we are doing, to have some preliminary work /exchange of ideas done before, like sample modules code, patterns and so on, that would make live discussions more focused on real, written grounds.

Al

[Jakov Sosic]

On 06/26/2013 07:27 PM, Alessandro Franceschi wrote:
> Good point.
> In the doc I placed some possible examples on where to assign names, but
> more (or remarks on the ones listed) cases are welcomed.
> (Let me have your email for an invitation to edit)
>
> Btw, comments and access to:
> https://docs.google.com/a/lab42.it/document/d/1D4OqEI5iuGJe63ODU91N6rnFKcxVAg1sAWbaQrGBlWA
> is open to everybody, and whoever writes modules should be somehow
> concerned.
> At the moment the discussion seems limited to very few people (even if I
> know various others are silently lurking :-)
>
> The same place of the document is to be considered temporary.

Let me just hop in and say that I'm really pleased with your stdmod
example, and I already did modify my first module in this way:

https://code.google.com/p/lutak/source/browse/tsm/manifests/init.pp

If you think I got something out of the order here, please post critique ;)



--
Jakov Sosic
www.srce.unizg.hr

[Alessandro Franceschi]

That's quite nice, I like the reduced verbosity of the code and essentiality of an all in one (init.pp)  location for resources.

For better reusability I'd provide a *_template option to manage the templates of  all the different files you manage, leaving as the default the currently hardcoded ones.

Also the backup_status and archive_status arguments follow an approach that ... erm.. was revisited in the current version of the  "ongoing standard".

Discussion about stdmod and naming standards seems faded again, and I've been quite distracted too.

I've had recently the need to make a Kibana (3) module and this is the result, for the moment, based on stdmod patterns.

https://github.com/example42/puppet-kibana

There are quite a lot other sample layouts, for different kind of modules that can be  done and shared.
Suppose is just a matter to show some works.

[Jakov Sosic]

On 08/06/2013 01:17 AM, Alessandro Franceschi wrote:

> That's quite nice, I like the reduced verbosity of the code and
> essentiality of an all in one (init.pp) location for resources.

Yeah, I don't like to partition my module if it's quite simple, as this
one is.

> For better reusability I'd provide a *_template option to manage the
> templates of all the different files you manage, leaving as the default
> the currently hardcoded ones.

Yeah, if I decide to publish the module on forge I will for sure add
those parameters too. We need to add Debian support in the house, and
after that I will maybe publish it.

TSM is a commercial product, we have in-house RPM/DEB packages (because
IBM provided ones are total shit), so without that packages module is
kinda useful.

This was my attempt to make the 'sample' module for our team to look at
when they code their own.

> Also the backup_status and archive_status arguments follow an approach
> that ... erm.. was revisited in the current version of the "ongoing
> standard".

Can you point me to the revisited approach? I really like the current one :D



--
Jakov Sosic
www.srce.unizg.hr

[Alessandro Franceschi]

Well, it makes sense to try to follow, a resource_parameter naming, where possible.

So to manage the service status (enable/ensure), probably is more natural to use two different parameters, like service_ensure and service_enable rather than one like service_status.
In your case the name of the parameters could be something like backup_service_enable archive_service_ensure (incidentally this would reduce even more the verbosity of your module).

The "current" status of the stdmod naming proposal is still :
https://github.com/stdmod/puppet-modules/blob/master/Parameters_List.md

there has been some concern about the amount of parameters or the need of some of them, but I think it makes sense to give standard names for different possible cases, even if you don't necessarily need to use them.

  

--
Jakov Sosic
www.srce.unizg.hr

[Jakov Sosic]

On 08/08/2013 09:10 PM, Alessandro Franceschi wrote:
> natural to use two different parameters, like service_ensure and
> service_enable rather than one like service_status.

Oh, I see. Simpler from the user perspective but more costly from
performance perspective...

> there has been some concern about the amount of parameters or the need
> of some of them, but I think it makes sense to give standard names for
> different possible cases, even if you don't necessarily need to use them.

Yeah, that interested me too... Imagine having like 20 modules like this
one included in a node manifest... Wouldn't the compile time explode
because of all the extra hiera lookups?! :-/


--
Jakov Sosic
www.srce.unizg.hr

[Alessandro Franceschi]

For sure the number of parameters has some impact , as the number of resources managed by the module.

Some Puppet performance experts ( Brice, are you reading :-? ) can surely give better ideas of the impact of them when used on scale.

I think it makes sense to make modules that provide some minimal, most common, reusability parameters and have the occasion to add features and parameters with predictable names and functions.
This would ease the progressive refinement of a module keeping a standard layout.

Really, we should provide and explore more examples, with different layouts for different modules structures, and see how this kind of parameters fit. 

my2c

 

--
Jakov Sosic
www.srce.unizg.hr

[David Schmitt]

Actually, I'm not much concerned about the compile-time performance,
given that the alternative are modules that cannot be re-used because
they encode narrow assumptions and do not provide all necessary
extension points.


Regards, David

[Alessandro Franceschi]

I've made some homework.

https://github.com/stdmod/puppet-wget

A wget module cloned from the existing openssh one.

The git history can show how easy and quick can be to make new modules (well this one is of course a very easy one) with the same parameters patterns. 

I've also gathered some sample modules which explore different usage cases here:

https://github.com/stdmod/puppet-modules/tree/master/modules

Feel free, anybody, to add new links to modules done with these (or proposed) patterns... (bin/clone.sh is your friend).

(*) We are still talking about a draft...  the current ones still need a few basic definitions ... ( package or package_name? file or file_path? or config? or config_path?... still here the current list https://github.com/stdmod/puppet-modules/blob/master/Parameters_List.md ) 

[devzero2000]

On Mon, Jun 17, 2013 at 1:17 PM, Matthias Saou <matt...@saou.eu> wrote:

Hi,

First off... is these some pun I'm not getting, or was it supposed to
be "Holy Grail"? Just wondering :-)

Also, when I read "Puppet Modules Standard naming conventions", I
thought it was just about module names, but it seems to be about
parameter naming inside all of the module resources.

After reading the shared Google document, I got reminded a lot of
what I've seen happen with RPM packaging over the years. Lot of
similarities, since both Puppet and RPM are tools which give a lot of
freedom in the way things get implemented.

Think of GNU/Linux distributions such as Fedora, RHEL, Mandriva, SuSE,
etc. which all use RPM packages as their building blocks : They are not
to be considered compatible for various reasons, yet they could have
been in theory.

RPM based distro storically have diverged mostly for brand and political reason. There are companies that sell support to the users who sponsor the developments. The same RPM was forked countless times with patches incompatible or not upstream. There have been propose in the past  to unify the packaging effort but failed http://lists.rpm.org/pipermail/rpm-maint/2008-June/002187.html.

RPM is a story in itself, believe me.I know it very well.

Puppet as a configuration management it is a completely different story in my opinion. I was possible to do more, and Francesco effort  goes in the right direction, in my opinion.

Best Regards

[Alessandro Franceschi]

An update on the stdmod works, which are going on slowly, but still moving.

We are getting near to define a basic set of names for modules parameters.
In particular there's a pull request where various discussions are ongoing on some basic naming patterns:

https://github.com/stdmod/puppet-modules/pull/1

the intention is to reach, with this PR, a point where at least the most used and important params names are defined and modules can start to be created / updated based on them.

If you don't like the proposed names or have suggestions to make, this is the right moment.

Please do that commenting directly the linked pull request.

Al