Layout and responsibility of stdmod modules

[Alessandro Franceschi]

Following a suggestion from Dolf I'm splitting the StdMod: Git, Branching, Versioning, Backwards Compatibility, Repos thread in two parts, 

the one about How to organize the repos, may go on on the original one where we can discuss git branching and modules management, here we can discuss about 'Layout and responsibility of stdmod modules' .

I copy here my starting (and effectively off topic) post and the relevant follow-up, for convenience.

Sorry if copy and paste from different posts which might mess up with dates, authors, and flow of discussion

Let's anyway discuss here about stdmod evolutions.

An active discussion of the parameters naming is ongoing here:
https://github.com/stdmod/puppet-modules/pull/1
feel free to add contextual comments as we're defining some last touches.

----------------------------------

On Sep 25, 2013, at 3:41 PM, Javier Bértoli <javier...@gmail.com> wrote:

El miércoles, 25 de septiembre de 2013 00:53:59 UTC-3, egeland escribió:

I for one think this is a great approach. Hopefully, Dolf agrees. :)

Should the all-in-one collection be based off the nextgen repo, maybe?

Or start from scratch?

I would start from nexgen modules. There are a lot of hours in coding in there that would be a pitty to throw away starting from scratch, don't you agree? :)

Happy to help where I can.

I think there's also a lot of room for improvement in the existing modules arising from the POV of someone who is testing and integrating them as an interdependent set. Thinking off the top of my head:

• Reviewing / extending / rewriting the documentation: many modules have undocumented features that I found only reading the code. That even happens with the modules I've written, as sometimes I don' t know how to explain things clearly and feel that the documentation I write is kind of poor :|
• Reviewing / extending / adding rspec tests for the modules. Many modules have defines or classes that are simply not being tested, or existing tests don't cover some use cases that you might need. Al have stated that he wanted to improve these testings, and in the process of integrating the ex42mods in a single "all-the-modules-you-might-want" repo (we need a name here :D) these tests will be a "must have", just to keep everything working OK.
• Request for features: particularly those related to interdependency.
• Submit patches? :)

 Any other ideas?

Let's move.

This is not something that I can do alone.

NextGen modules are there, somehow stable, doing their work.

They are requiring a lot of support that I can't give in my free time.

Even for testing submitted patches.

We have to start to work on brand new 3.x modules and a sane ecosystem.

I suggest these principles:

0- No NextGen legacies 

1- Real single point of responsability for modules

2- Higher abstraction  stack modules

3- Follow stdmod naming conventions

4- Distributed development (modules maintaners)

5- Implementation and usage patterns coherency

0- No NextGen legacies

The current NextGen modules should not be taken as reference, no needs to have backwards compatibility.

Let's take the good things (defines, implementation patterns, correct params and setups for different OS ) and build from them.

Let's cut all the parameters and functionalities that don't fit with points 1 and 3.

1- Real single point of responsability for modules

This is important and critical for the whole long term feasibility and reusability of the modules:

an application module should only expose what's needed to configure and manage its application.

No relation to third modules, no options to provide implementation logics of higher abstractions.

For example, an apache main class should expose just the essential parameters to just manage (in a reusable way) the application.

Then the module may have custom defines to manage modules, virtualhosts, users, or whatever.

We can reuse the good ones around, adapting the parameters to stdmod conventions.

It might even have something like apache::passenger, with its own parameters, but is should not expose any passenger related parameter in the main class.

I would add to these basic parameters the ones need to firewall and monitor them, keeping them in the most simple and open way, just 2 parameters:

$monitor_class - The name of the class to use to manage monitoring.

$monitor_config_hash - An open to any option configuration hash (we may stanrdardize it when working on the new monitor / firewall implementations ) 

Same for $firewall_

I'd also provide in each module (that needs them) standard "general purpose" defines to:

- Manage single configuration files (useful when multiple config files are needed and you don't want to clutter the main class params to manage them)

- Manage single configuration entries (via augeas?)

No read need, IMHO, to have different defines with specific parameters for different configuration files, just assure there's a "catchall" config_hash parameter.

So just dumb interfaces to the application's configuration , all the logic and the implementation details can be provided by 2:

This would allow quick creation of basic modules with at least these "minimal" features, then we can add or import more specific ones. 

Here we have to place rspec tests, rspec-system tests and essential documentation (all the stdmod standards related one can be avoided, imho)

2- Higher abstraction  stack modules

Here we move all the logic to manage modules interdipendences and integrations.

So we might have modules like:

puppet-stack_lamp

puppet-stack_liferay (but also just puppet-liferay as long as it follows stacks' patterns)

puppet-stack_rails

puppet-stack_logs ( here's a sample https://github.com/example42/puppet-stack/blob/master/manifests/logs.pp )

puppet-stack_puppetinfrastructure

Stacks do, in some way, what we often do in our puppet setups (and have called them roles, hostgroups, profiles) exposing higher level parameters that allow and manage the logic provided by the stack to integrate different application modules.

They are supposed to be used by nodes doing different things which are needed to build the stack.

In stacks some parameters could be managed in automatic ways, based on the nodes who use their single components.

With puppetdbquery some wonderful things could be done.

The parameters exposed by a stack class should be about:

- if to include a specific application in the node ( install_webserver ?)

- what to use for each stack's application (where alternatives are provided) ( webserver_name ? ),

- how to configure them (parameters hash, custom template files )

- endpoints, credentials, urls…

Stacks are the place for integration tests. 

3- Follow stdmod naming conventions

They are still not definitive, I really urge whoever is interested in a good shared modules ecosystem to contribute actively to https://github.com/stdmod

We definitively need to decide on few remaining things, IMHO, like:

file (file_path? config_file? config?) (1)
file_source (source? config_file_source? config_source?)
file_template (template? config_file_template? config_template?...)

Since we are doing this as users of Ex42 modules we might decide to choose the current names for the most used parameters (template? source?) as long as they make sense in the whole stdmod picture.

4- Distributed development (modules mantainers)

This can't be an Example42's only work. I can't and don't want to manage multiple dozens of modules of applications I don't know or use.

Once good blueprints are available for different kind of modules, we can quickly create new modules based on them, adding and adapting the available defines, facts, functions from nextgen and  other open source modules.

Possibly involving the relevant authours, also as maintainers.

Code can stay in each mantainer's repos , Dolf, I, whoever can manage their own collections of stdmodules and stacks, pointing to gitsubmodules at the desired version / head (or managing them in a single big git repo)

Of course just one "stdmod" module for application would be the ideal: forks just for pulling back to upstream, as managed by the official maintainer(s).

All this, strictly following point 5:

5- Implementation and usage patterns coherency

It's very important to keep coherency inside the modules set, both for naming and usage patterns.

For this the sooner the finalize naming standars and we have good samples and  templates, the better.

I would stick the code of the modules for Puppet 3 compatibility, no need to do now things that support older versions and too early to embrace fully Puppet 4 ( > 3.3) only  features (data in modules, future parser, containers… ).

They will be useful probably to manage in a smart way firewalling and monitoring, but given the simple basic lay

For this reason I would leave open parameters like monitor_class or monitor_config_hash that may adapt to any future evolution.

Next steps…

- propose blueprints for common  module's layout.

- make quick and easy common usage, single package modules (to just "include"): zip, wget, git…

- make essential modules (to improve with time) for the most cross used apps: apache, nginx, mysql…

- explore stacks

my2c

Al

-------------------------

On Monday, October 7, 2013 4:50:01 PM UTC+2, javier wrote:

On 10/02/2013 04:58 AM, Alessandro Franceschi wrote: 

Guys,

some kind of feedback here is definitively welcomed.

Before starting to rollout new "stdmod" modules there are some points to define, both about naming (on which I don't want to be the only decision-maker) and about approach.

The params naming as in https://github.com/stdmod/puppet-modules/blob/master/Parameters_List.md is almost defined, but there are still 2 key decisions to make:

1- Name of the parameter that maps to the namevar: For example: package vs package_name, file vs file_path, service vs service_name. The are reasons for both the alternatives, currently PuppetLabs modules tend to follow the "package_name" alternative, while Example42's NextGen ones the "package" one.

2- Name of the main configuration file managed by a class: file (currently used is stdmod sample modules)  vs config_file (currently used in Ex42 NextGen) vs config

3- Name of the parameter that refers to template or source to use for the main configuration file:  file_template (currently used is stdmod sample modules)  vs template (currently used in Ex42 NextGen) vs config_file_template vs config_template (same for source)

Decision of these names is important both for the whole coherency of the naming standards and for the possibility to start to work on the new modules, it's a required prerequisite.

Sent you a PR with some ideas. 

Seen it, let's move there the discussion about those details.

Also would like to have your idea about the currently not decided namings (the params where you find some alternatives).

 

> Another point is about the opportunity to consider to use some of PuppetLabs 
>  modules for some core applications: for example apache, mysql, postgresql, 
> firewall. Both because they are quite good and complete and because they are 
>  widely used and their adoption directly in the stdmod set would reduce many 
>  incompatibility issues. The downside here is that they are not (still?) 
> fully stdmod compliant, but I suppose that can be worked on with time, 
> proposing PR to the core maintainers. The same approach could be done for 
> other largely used and recognized modules from other authors: key points here 
> are better interoperability and more code reuse, at eventually the cost of 
> the general coherency of the modules set. What do you think about this? 

I use a couple of modules from puppetlabs, but others I found better to rewrite 
them using their logic but following ex42 modules structure. Haven't tried 
particularly those that you propose, but I think it would be easier to: 

   1. Write a ex42 scaffold 
   2. Add code following already existing modules (even using c&p and giving 
attribution where needed?) 

   This approach would give us more coherence, but will be more work. Really 
don't have a defined preference here, so it's possible you can make me change my 
mind easily :) 

Well there are pros and cons:

redoing modules from an ex42 we gain in coherency and we are more free to do them in the way we prefer , but we lose in interoperability , as some modules like puppetlabs' apache , postgresql and maybe firewall are becoming a sort of standard for most cases.

To merge their best parts (and adapt them to stdmod standards) would be an option , but it would create yet another fork and I'd rather prefer to involve other modules authors to embrace stdmod and use in the ex42 set the best and  most used modules for the common applications (where interoperabiliy matters more).

 

> Another point is about the opportunity to consider to use some of PuppetLabs modules for some core applications: for example apache, mysql,  postgresql, firewall. Both because they are quite good and complete and because they are widely used and their adoption directly in the stdmod set would reduce many incompatibility issues.

> The downside here is that they are not (still?) fully stdmod compliant, but I suppose that can be worked on with time, proposing PR to the core maintainers.

> The same approach could be done for other largely used and recognized modules from other authors: key points here are better interoperability and more code reuse, at eventually the cost of the general coherency of the modules set.

> What do you think about this?

 

> Opinions definitively welcomed. 

   Another issue I came across these days: 
https://github.com/puppetlabs/puppetlabs-haproxy/pull/52#issuecomment-25638694 

--- 
While I appreciate this we don't plan to extend support of haproxy to 2.6, which 
is long past EOL. I'm afraid I'm going to reject this one. 
--- 

   So we probably should also consider dropping 2.6 support on ex42 modules (ie. 
removing it from tests, travis, etc.?) 

Yes, we can definitivley drop support for 2.6 in the modules 3.x and I would stick to current parser if there are not good reasons to opt, in some cases, for the future parser.

 

There's also another issue that I would propose to consider in the new modules structure:

• Adding a "/doc(s)" subdirectory in the module structure, where we put all that documentation that we are keeping currently at the top of each recipe.
• Trim down all the documentation at the top of each recipe to a ONE-LINER (two lines tops) explaining briefly the parameter options.

Much of the headers, nowadays, are repeated from module to module, with changes that were added to a module's file but not passed to another module, leading to inconsistencies of documentation between them

Keeping the "general modules' parameters documentation" in a separate file can help, ie., for an author to pull changes for these particular files under the "/doc(s)" dir, diff them against her module's copy of them and merge/modify/update them.

Also, it will make easier to add examples where needed or extend the documentation as desired by the module's author.

What do you think?

Good idea.

I too would prefer to slim down the in class documentation, the sample stdmod modules already cut the documentation for common params, this would be an extra cut.
Puppetdoc has received few love in the last years on the Puppet side, so we might just plainly decide to place just the 2 lines headers that are needed for puppet-lint compliance.

Javier, I've added you as owner on  https://github.com/stdmod , feel free to add sample modules blueprints based on the latest stdmod naming we have discussed here https://github.com/stdmod/puppet-modules/pull/1 (comments from others are welcomed) . I'll have to adapt the existing stdmod sample modules, once we have a decent set of different modules types we can actively (and quickly?) start to make the 3.x modules cloning from existing modules and adding defines, tests, libraries and docs as needed.

Dolf is doing a terrific work on 3.x iptables and firewall modules (check out at https://github.com/example42/puppet-iptables/tree/devel andhttps://github.com/example42/puppet-firewall/tree/devel) and that's one of the prerequisites for a large migration to 3.x modules (the monitoring stuff is another one).

[Freeaqingme]

Hi,

Thanks for splitting the thread.

I agree with the principles you've outlined in your email in general. They seem very reasonable to me.

Some remarks though:

> 0- No NextGen legacies

It seems a very noble goal, and will be doable in a lot of places. However, should we also apply this to modules that are really commonly used as dependencies? The example42/puppet-firewall module coms to mind. If we were to break BC there, one couldn't use both Example42 as well as StdMod modules. Forcing people to stick with Example42 modules, or forcing us to ensure we port it all to StdMod.

> 1- Real single point of responsability for modules

> It might even have something like apache::passenger, with its own parameters, but is should not expose any passenger related parameter in the main class.

This seems a little contradictory to me? I'd say that we should strive for a single point of responsibility for _classes_. This means that for Apache, we could have these classes:

- apache: Manages Apache itself (some config stuff only, perhaps monitoring of the Apache instance)

- apache::install: Manages installation of Apache. This may be a bit overabundant for an application that can be installed by just 1 package statement, but some stuff needs compilation, etc. And it's easy for people to override.

- apache::repo: Manages the application repo for Apache

- apache::vhost: Configures a vhost, adds the relevant monitoring and firewall rules (proxies it to $monitor_class, $firewall_class)

- apache::module::passenger: Does the Passenger stuff

> $monitor_class - The name of the class to use to manage monitoring.

> $monitor_config_hash - An open to any option configuration hash (we may stanrdardize it when working on the new monitor / firewall implementations ). Same for $firewall_

Out of curiosity, how would you include a defined type ($monitor_class should probably be $monitor_type instead?) based on a config hash?

> 2- Higher abstraction stack modules

I like the idea. Question is though if we should put all the stacks we have in 1 repo, or should we use 1 repo per stack? Assuming you've given this some thought, what are your considerations? I don't use the module system from the forge myself, but if we put it all in 1 module, we can't use the Modulefiles here to manage dependencies.

As a sidenote, all stacks we do provide should provide means to set up distributedly. A LAMP stack is fun on one device, but it should provide the ability to setup Apache+PHP on one node, and MariaDb on another node (perhaps automatically configuring fw rules etc in the mean time?). Another example is the logs stack (which I wasn't aware of); it should be able to run rabbitmq on one node, elastic search on another (cluster of) node(s), etc.

> 3. Naming conventions

It was mentioned somewhere in this thread; I think we should strive to adhere to the same conventions as PuppetLabs does. Consistency can only benefit the Puppet ecosystem as a whole. Of course we can deviate from specific points, but that should be accompanied with some good argumentation, not just personal preference.

> 4- Distributed development (modules mantainers)

[...]

Agreed. Though I do like the thought of Al fixing all my fuckups :D (no worries, just kidding).

A factor that comes to mind here though is the distros and OS's we want to support. I myself usually use Ubuntu, and I think it's tough to expect to expect each contributor to be able to support Solaris through BSD through Linux (with who knows how many distros) through Windows (what versions?). I think we should define some versions and distro's we'd like to support (CentOS 6, Ubuntu 12.04) in all modules, and that all contributors should try to support at least those two. That can then be accompanied by some Rspec tests (should those be mandatory?), and we could configure Travis to test on all the OS's we try to support(? - I don't really know Travis).

Furthermore Al, I believe you have some Vagrant boxes that you use to test stuff on, assuming you've accumulated some best practices with those, could you perhaps share those? That would allow people to more easily test a variety of distro's.

Reason for putting the version stuff under 'distributed maintainers' is that many developers base their work off of 1 distro only, but we could just as well make it a separate principle of StdMod.

> Code can stay in each mantainer's repos 

What do you mean by this point? I'd personally like to have the upstream module reside under the StdMod organization for multiple reasons:

- People know where to file a PR under: That of the StdMod organization.

- There's a shared sense of ownership. Even though each module has its own maintainer, maintainers can be busy, on holidays, or otherwise (temporarily) awol. Other contributors should feel free to work on a module, even though they are not responsible for maintaining it.

- I've noticed that search engines tend to show up the original (parent) repo only in their results. SEO-wise it helps to host the original repos. When people see 'StdMod they'll hopefully use it more quicker than each contributor's individual name. Given that StdMod as a collection will be faster establishing a recognized brand.

- I like to mess with my own repos, without messing with any of the official ones.

Along those lines, I think it'd be good if all code that goes into any of the StdMod repo's is peer reviewed (by filing a PR). Not because I don't trust other contributors, but because two pair of eyes simply catch more than one pair of eyes. 

> 5- Implementation and usage patterns coherency

Agreed. I'd propose to have a base module that we can derive all other modules from. I know it's already there, but what I'd like to stress is the importance of having a single template only. With Example42 there are multiple templates, and I've seen that lead to inconsitencies, because different people used different templates (within the previous company I worked at).

Thoughts?

Dolf

-- Freeaqingme

[Alessandro Franceschi]

On Thursday, October 10, 2013 4:13:18 PM UTC+2, Freeaqingme wrote:

Hi,

Thanks for splitting the thread.

I agree with the principles you've outlined in your email in general. They seem very reasonable to me.

Some remarks though:

> 0- No NextGen legacies

It seems a very noble goal, and will be doable in a lot of places. However, should we also apply this to modules that are really commonly used as dependencies? The example42/puppet-firewall module coms to mind. If we were to break BC there, one couldn't use both Example42 as well as StdMod modules. Forcing people to stick with Example42 modules, or forcing us to ensure we port it all to StdMod.

Still have to figure out how to personally face the ex42 dependencies.
I for example would gladly add firewall/monitor and puppi integrations, and provide them as example modules / classes, the possibility to provide params that allow usage of custom classes for these ( $monitor_class, $firewall_class ) could allow general stdmod modules and third party's integrations. 

 

> 1- Real single point of responsability for modules

> It might even have something like apache::passenger, with its own parameters, but is should not expose any passenger related parameter in the main class.

This seems a little contradictory to me? I'd say that we should strive for a single point of responsibility for _classes_. This means that for Apache, we could have these classes:

- apache: Manages Apache itself (some config stuff only, perhaps monitoring of the Apache instance)

- apache::install: Manages installation of Apache. This may be a bit overabundant for an application that can be installed by just 1 package statement, but some stuff needs compilation, etc. And it's easy for people to override.

- apache::repo: Manages the application repo for Apache

- apache::vhost: Configures a vhost, adds the relevant monitoring and firewall rules (proxies it to $monitor_class, $firewall_class)

- apache::module::passenger: Does the Passenger stuff

Probably we are not saying different things, be it apache::passenger or apache::module::passenger, this class should have its own specific (stdmod style) params and the main apache class should not refer to it or having passenger specific params, because the logic of how to use apache and passenger can be better managed in a stack class.

 

> $monitor_class - The name of the class to use to manage monitoring.

> $monitor_config_hash - An open to any option configuration hash (we may stanrdardize it when working on the new monitor / firewall implementations ). Same for $firewall_

Out of curiosity, how would you include a defined type ($monitor_class should probably be $monitor_type instead?) based on a config hash?

My idea is: 

$monitor_class is the name of the class to use (ie: example42::monitor::apache 
and $monitor_config_hash is an hash where you can place whatever data that the module has to consume.
So the module should interpret and manage the config hash.
It could be interesting also to automatically provide , internally, an extra hash with all the scope values, so that the module data (with stdmod naming) can an automatically provided to the class.
This idea actually is 5 minutes fresh, in a stdmod openssh module ther could be something like:

  if $openssh::monitor_class {
    class { $openssh::monitor_class:
      config_hash => $monitor_config_hash,
      scope_hash  => inline_template('<%= scope.to_hash.reject { |k,v| k.to_s =~ /(uptime.*|path|timestamp|free|.*password.*|.*psk.*|.*key)/ } %>'),
    }
  }

 
So in the monitor class you have to accept these params and can manage them as you want.
The nice thing here, is that in $scope_hash  (where maybe should be placed vars only from the class scope and not the whole scope) you have all the effective openssh  params and they are useful for any kind of operations.

> 2- Higher abstraction stack modules

I like the idea. Question is though if we should put all the stacks we have in 1 repo, or should we use 1 repo per stack? Assuming you've given this some thought, what are your considerations? I don't use the module system from the forge myself, but if we put it all in 1 module, we can't use the Modulefiles here to manage dependencies.

Yes, one repo / module for stack.

Many stacks from different authors, possibly with standardized params.

 

As a sidenote, all stacks we do provide should provide means to set up distributedly. A LAMP stack is fun on one device, but it should provide the ability to setup Apache+PHP on one node, and MariaDb on another node (perhaps automatically configuring fw rules etc in the mean time?). Another example is the logs stack (which I wasn't aware of); it should be able to run rabbitmq on one node, elastic search on another (cluster of) node(s), etc.

Sure, the pattern that the stack class is used by many differnet nodes, providing different params according to their functions.
Looks at this sample :
https://github.com/example42/puppet-stack_puppet/blob/master/manifests/init.pp
here you have params like:

  $puppet_install                   = true,
  $puppetdb_install                 = false,
  $postgresql_install               = false,

so that the stack class can configure using simply modules and providing predefined templates (which can be overridden by users with params like:

  $puppet_config_template           = 'stack_puppet/puppet/puppet.conf',
  $puppet_options_hash               = { },

 
the logic and the glue is all inside the stack class, here you can have declarations like: apache::modules::passenger
and here, you can have also mechanism of autoscaling using the deep knowledge of puppetdb.
Just look at these samples (no idea if they work, but that's the concept:

  $activemq_servers = query_nodes('Class[activemq]')
  $activemq_servers_ip = query_nodes('Class[activemq', ipaddress)

 $puppetmaster_servers = query_nodes('Class[puppet::server]')
 $puppetmaster_servers_ip = query_nodes('Class[puppet::server]', ipaddress)

Laying at an higher abstraction layer, the stack classes have wider visibility on the infrastructure's cofngiuration and they can accordingly configure the single nodes.
 

> 3. Naming conventions

It was mentioned somewhere in this thread; I think we should strive to adhere to the same conventions as PuppetLabs does. Consistency can only benefit the Puppet ecosystem as a whole. Of course we can deviate from specific points, but that should be accompanied with some good argumentation, not just personal preference.

Naming conventions on PuppetLabs modules has been far from being coherent, only recently the module team has started to standardize following more or less common naming.
Many of these are already present in stdmod naming , other not, either because not used or because not coherent with the whole picture.
At least in the opinion mine and of who is contributing with suggestions.
 
 

> 4- Distributed development (modules mantainers)

[...]

Agreed. Though I do like the thought of Al fixing all my fuckups :D (no worries, just kidding).

A factor that comes to mind here though is the distros and OS's we want to support. I myself usually use Ubuntu, and I think it's tough to expect to expect each contributor to be able to support Solaris through BSD through Linux (with who knows how many distros) through Windows (what versions?). I think we should define some versions and distro's we'd like to support (CentOS 6, Ubuntu 12.04) in all modules, and that all contributors should try to support at least those two. That can then be accompanied by some Rspec tests (should those be mandatory?), and we could configure Travis to test on all the OS's we try to support(? - I don't really know Travis).

Stmod modules are about reusability and the first rule of reusability is support to different OS.
I personally think that they should support at least RedHat family 6, Debian 6 and 7 , Ubuntu 12.04
Better if they also support RedHat 5, and as mush as possible other oses.
Tests should be present, I would not *enforce*  full coverage though.

 

Furthermore Al, I believe you have some Vagrant boxes that you use to test stuff on, assuming you've accumulated some best practices with those, could you perhaps share those? That would allow people to more easily test a variety of distro's.

I use the https://github.com/example42/puppet-playground  

Reason for putting the version stuff under 'distributed maintainers' is that many developers base their work off of 1 distro only, but we could just as well make it a separate principle of StdMod.

Well here we have to consider 2 cases:
- modules based on stdmod naming standards (the can support whatever the author wants)
- "official" stdmod modules (if we will ever have them), in these case they should adhere to some minimal requirements, about OS support minimal params (like the ones about monitoring etc) and so on.

> Code can stay in each mantainer's repos 

What do you mean by this point? I'd personally like to have the upstream module reside under the StdMod organization for multiple reasons:

- People know where to file a PR under: That of the StdMod organization.

- There's a shared sense of ownership. Even though each module has its own maintainer, maintainers can be busy, on holidays, or otherwise (temporarily) awol. Other contributors should feel free to work on a module, even though they are not responsible for maintaining it.

- I've noticed that search engines tend to show up the original (parent) repo only in their results. SEO-wise it helps to host the original repos. When people see 'StdMod they'll hopefully use it more quicker than each contributor's individual name. Given that StdMod as a collection will be faster establishing a recognized brand.

- I like to mess with my own repos, without messing with any of the official ones.

I suppose this will evolve on how people will make it evolve.
I can embrace both a situation where single authors make their own stdmod modules and in the github stdmod repo there is just a selection of a module per application and the links in the relevant external repos o have the official stdmod modules directly under the stdmod account.
 

Along those lines, I think it'd be good if all code that goes into any of the StdMod repo's is peer reviewed (by filing a PR). Not because I don't trust other contributors, but because two pair of eyes simply catch more than one pair of eyes. 

Sure. This actually can apply mostly in the second case.
 

> 5- Implementation and usage patterns coherency

Agreed. I'd propose to have a base module that we can derive all other modules from. I know it's already there, but what I'd like to stress is the importance of having a single template only. With Example42 there are multiple templates, and I've seen that lead to inconsitencies, because different people used different templates (within the previous company I worked at).

The internal structure of the modules (at least teh "official" stdmod ones should be coherent, but thi doesn't mean having just one template to use as "golden image". I'd rather make different templates for different kind of modules.
- A template for the standard package/service/config file module
- A template for a simpler package/config module
- A template for multiple services/packages applications 
- A template for java applications and so on

Thoughts?

These ones.
al
 

[Freeaqingme]

Hi Al,

I think for the biggest part we agree. Some minor things though;

> Well here we have to consider 2 cases:
> - modules based on stdmod naming standards (the can support whatever the author wants)
>- "official" stdmod modules (if we will ever have them), in these case they should adhere to some minimal requirements, about OS support minimal params (like the ones about monitoring etc) and so on.

I'd prefer to only have "official" stdmod modules in the StdMod organizations. Meaning that people should be able to simply pick a module from the StdMod organization, and know what support and level of quality they can expect. As part of that coherency. Maybe that's what you already meant, but I'd like to make sure ;)

> My idea is:  $monitor_class is the name of the class to use (ie: example42::monitor::apache 

> and $monitor_config_hash is an hash where you can place whatever data that the module has to consume.

I've replied to this separately on the Puppet Users Google Groups list.

>> Along those lines, I think it'd be good if all code that goes into any of the StdMod repo's is peer reviewed (by filing a PR). Not because I don't trust other contributors, but because two pair of eyes simply catch more than one pair of eyes.
> Sure. This actually can apply mostly in the second case.

What second case are you referring to? I think that it'd be good that for all code merged into any repo in the StdMod a Pull Request would be the preferred way.

> The internal structure of the modules (at least teh "official" stdmod ones should be coherent, but thi doesn't mean having just one template to use as "golden image". I'd rather make different templates for different kind of modules.
> - A template for the standard package/service/config file module
> - A template for a simpler package/config module
> - A template for multiple services/packages applications 
> - A template for java applications and so on

The problem I foresee with a distinction based on the 'standard' v. 'simpler' module is that simple modules will eventually grow, and become extra-large. That's not a problem, but it will cause people to deviate from the standard. I'm all for making specific modules, but only for specific use cases. As such, I can imagine having a 'default' template, and a 'stack' module for example. But things based on size, simplicity, or the fact that they are published on the forge I really would like to recommend against.

Nonetheless, I'm happy to conclude that for 9 out of 10 points we seem to be on the same line.

Regards,

Dolf

-- Freeaqingme

On Thursday, October 10, 2013 1:43:48 AM UTC+2, Alessandro Franceschi wrote:

[Alessandro Franceschi]

Il giorno mercoledì 16 ottobre 2013 17:30:05 UTC+2, Freeaqingme ha scritto:

Hi Al,

I think for the biggest part we agree. Some minor things though;

> Well here we have to consider 2 cases:
> - modules based on stdmod naming standards (the can support whatever the author wants)
>- "official" stdmod modules (if we will ever have them), in these case they should adhere to some minimal requirements, about OS support minimal params (like the ones about monitoring etc) and so on.

I'd prefer to only have "official" stdmod modules in the StdMod organizations. Meaning that people should be able to simply pick a module from the StdMod organization, and know what support and level of quality they can expect. As part of that coherency. Maybe that's what you already meant, but I'd like to make sure ;)

Let's do things at steps.

At the beginning I'd encourage both stdmod "official" modules, with multiple maintainers and all the necessary peer review, and "branded" stdmod based modules, where single authors are free to manage indipendently their modules and experiment, following stdmod conventions, and iterate more quickly.

When we will see many different people doing their own stdmod version of a module we might decide to merge them and maintain an official one.

Under this point of view it makes sense to start to consider official stdmod repos for common applications like apache, nginx, mysql, postgresql, puppet, apt, yum and other widely used ones.

I prefer to avoid to place too many rules (if not for the naming standards) at the beginning... let's encourage development before trying to regulate it.  

 

> My idea is:  $monitor_class is the name of the class to use (ie: example42::monitor::apache 
> and $monitor_config_hash is an hash where you can place whatever data that the module has to consume.

I've replied to this separately on the Puppet Users Google Groups list.

>> Along those lines, I think it'd be good if all code that goes into any of the StdMod repo's is peer reviewed (by filing a PR). Not because I don't trust other contributors, but because two pair of eyes simply catch more than one pair of eyes.
> Sure. This actually can apply mostly in the second case.

What second case are you referring to? I think that it'd be good that for all code merged into any repo in the StdMod a Pull Request would be the preferred way.

> The internal structure of the modules (at least teh "official" stdmod ones should be coherent, but thi doesn't mean having just one template to use as "golden image". I'd rather make different templates for different kind of modules.
> - A template for the standard package/service/config file module
> - A template for a simpler package/config module
> - A template for multiple services/packages applications 
> - A template for java applications and so on

The problem I foresee with a distinction based on the 'standard' v. 'simpler' module is that simple modules will eventually grow, and become extra-large. That's not a problem, but it will cause people to deviate from the standard. I'm all for making specific modules, but only for specific use cases. As such, I can imagine having a 'default' template, and a 'stack' module for example. But things based on size, simplicity, or the fact that they are published on the forge I really would like to recommend against.

Well, take a wget module.
You know that it will never have a service.

It makes sense to start it from a package-config skeleton and then add defines and custom stuff as you would do for any module.

To create it from a common package-service-config skeleton would involve, first remove (manually) all the references to the service, then add all the custom wget stuff.

I'd make skeletons for these different modules types, the most common ones.

 

Nonetheless, I'm happy to conclude that for 9 out of 10 points we seem to be on the same line.

Good!

Al