> I would like to anticipate you the "Lab42 Puppet Infrastructure" a
> growing set of recipes, freely usable, with these goals:
as several people are coming up more and more with publishing their
modules, we should begin to think about a "central" community driven
platform for modules.
DavidS published already some modules, me as well ours, which might be
forks of DavidS one's as well some own new modules and their are a lot
of others as well.
So after some prework and proposal of DavidS on
http://reductivelabs.com/trac/puppet/wiki/CommonModules I would propose
to continue with his proposal and try to figure out how we could do some
kind of community place for modules which won't get cluttered into
x-different module versions which won't be compatible to each another.
So after some discussion yesterday with lak and someone else (sorry lost
my backlog :-/ ) I would note the following points:
- a central wiki page which would contain all modules inside this
community process
- a central git-repository (proposal was to use github.com) with one
maintainer for each module.
- possibilitities to add other development git-repos for the specific
module on the wiki on a sublevel. which are forks of this specific modules
- avoid cluttering of modules, the goal is to get one module which
should fit for nearly all people. (might be an utopia ;) ) and at the
end there shouldn't be in my opinion for example a dozen different
modules for apache.
- documentation: every module should fit some documentation standards we
define and each maintainer tries to achieve them for their module.
- free license: huh let's start another flamewar :P but what i'd like to
say is that the modules should be published in the sense of free
software (so bsd or gpl) and it should be possible to use them each
another. i don't care which license the specific module have, the
important thing is that they are free and can be used along each another.
- definition of dependencies: if modules use each another they should
document their dependencies. there was once a discussion about how that
could be done, but i don't know how far this is yet.
so this are my notes on that, any other thought ideas?
greets pete
Peter Meier wrote:
> So after some prework and proposal of DavidS on
> http://reductivelabs.com/trac/puppet/wiki/CommonModules I would propose
> to continue with his proposal and try to figure out how we could do some
> kind of community place for modules which won't get cluttered into
> x-different module versions which won't be compatible to each another.
This is going to be interesting- some thoughts:
- - how do you handle the issue of distribution/platform inter-operability
in a consistent manner?
- - Also how do you handle cross-module testing, inter-relationships and
compatibility? You need to consider some form of testing methodology or
even test scripts to ensure updates to some modules don't break other
modules.
> - a central wiki page which would contain all modules inside this
> community process
Happy to help with this.
> - a central git-repository (proposal was to use github.com) with one
> maintainer for each module.
Hmm - I think you'd need to consider more than one maintainer - easier
to ensure a module isn't orphaned. Perhaps a group who look after
collections of modules.
> - possibilitities to add other development git-repos for the specific
> module on the wiki on a sublevel. which are forks of this specific modules
What would the purpose of the forks be?
> - avoid cluttering of modules, the goal is to get one module which
> should fit for nearly all people. (might be an utopia ;) ) and at the
> end there shouldn't be in my opinion for example a dozen different
> modules for apache.
Maybe a core module and sub-modules by OS?
> - documentation: every module should fit some documentation standards we
> define and each maintainer tries to achieve them for their module.
Agreed - happy to help develop the standards.
> - free license: huh let's start another flamewar :P but what i'd like to
> say is that the modules should be published in the sense of free
> software (so bsd or gpl) and it should be possible to use them each
> another. i don't care which license the specific module have, the
> important thing is that they are free and can be used along each another.
It'd be nice if the license was consistent with Puppet - GPLv2 - but I
guess not critical.
> - definition of dependencies: if modules use each another they should
> document their dependencies. there was once a discussion about how that
> could be done, but i don't know how far this is yet.
This is a function of a module README. I think every module needs one
of these - best based on a template that ensures all required
information is available for the module.
> so this are my notes on that, any other thought ideas?
Sounds like a great concept. Be good to hear from all the current
module maintainers.
Cheers
James
- --
James Turnbull (ja...@lovedthanlost.net)
- --
Author of:
- - Pulling Strings with Puppet
(http://www.amazon.com/gp/product/1590599780/)
- - Pro Nagios 2.0
(http://www.amazon.com/gp/product/1590596099/)
- - Hardening Linux
(http://www.amazon.com/gp/product/1590594444/)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFIB05Q9hTGvAxC30ARAmhDAKCP0HX7IKJSmDLqw0zQVJxN2vg5CACgwrXY
W3VWO3jSwrhgjiFohtFh0oM=
=hO6I
-----END PGP SIGNATURE-----
> Is there a general bias towards git in the puppet community? I haven't
> used it. We use subversion for our puppet manifests because that is
> what the web team uses for content and development version control. I
> have a number of modules I would like to share with the puppet
> community, but because the Common Modules page refers to git, I don't
> particularly want to set up a separate repo just for this purpose.
The probable reason for the apparent git bias is because I somewhat
arbitrarily decided to switch Puppet to git.
It might make sense to support other SCM tools, but I can't recommend
highly enough that only distributed version control tools be used. In
particular, git's support for in-repository branches is really just
fantastic, I can't say enough good about it.
For modules especially, where lots of people will have various small
modifications to a module, it makes a lot of sense.
--
He played the king as if afraid someone else would play the ace.
--John Mason Brown
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com
> Is there a general bias towards git in the puppet community? I haven't
> used it. We use subversion for our puppet manifests because that is
> what the web team uses for content and development version control. I
> have a number of modules I would like to share with the puppet
> community, but because the Common Modules page refers to git, I don't
> particularly want to set up a separate repo just for this purpose.
i could provide some ruby script i use to dump, on a half-daily basis,
our modules from one subversion repo into different git-repos.
it's just that sharing your code on a horizontal, community and
distributed manner is with git very easy. as well to merge different
sources of modules (branches) together.
greets pete
See my suggestion for a metadata file for modules that would help
define much of this:
http://groups.google.com/group/puppet-users/browse_thread/thread/66b90ee0e3a229c2?hl=en
-Blake
> - Also how do you handle cross-module testing, inter-relationships and
> compatibility? You need to consider some form of testing methodology or
> even test scripts to ensure updates to some modules don't break other
> modules.
I agree that this point is quite hard. And looking at the current state
of writing modules that there is (yet) no way to write module
documentation not module depenecies nor tests, it might be something
that can evolve with the growing module repository? do we really have
this from the beginning on?
>> - a central wiki page which would contain all modules inside this
>> community process
>
> Happy to help with this.
How about to start on wiki:CommonModules to document the process an idea
a bit more. I really like the work already done by DavidS, it helped me
a lot! (so thanks!!) and to start wiki:CommonModulesList and try to get
a starting point?
Maybe it would be good to start with one (rather simple and easy)
module, to see that there a plenty of people with different ideas :)
>> - a central git-repository (proposal was to use github.com) with one
>> maintainer for each module.
>
> Hmm - I think you'd need to consider more than one maintainer - easier
> to ensure a module isn't orphaned. Perhaps a group who look after
> collections of modules.
>
>> - possibilitities to add other development git-repos for the specific
>> module on the wiki on a sublevel. which are forks of this specific modules
>
> What would the purpose of the forks be?
this was actually the idea of this kind of group of maintainers, but I
agree that it might be better to have more than one which have push
access to the official one. This one should be treated as the stable and
working one. All the others might be forks or modules in development and
therefor to bee seen as rather unstable.
>> - avoid cluttering of modules, the goal is to get one module which
>> should fit for nearly all people. (might be an utopia ;) ) and at the
>> end there shouldn't be in my opinion for example a dozen different
>> modules for apache.
>
> Maybe a core module and sub-modules by OS?
yeah maybe. however the core-module should do one every OS the same.
>> - documentation: every module should fit some documentation standards we
>> define and each maintainer tries to achieve them for their module.
>
> Agreed - happy to help develop the standards.
might be a starting point for the wiki page.
>> - free license: huh let's start another flamewar :P but what i'd like to
>> say is that the modules should be published in the sense of free
>> software (so bsd or gpl) and it should be possible to use them each
>> another. i don't care which license the specific module have, the
>> important thing is that they are free and can be used along each another.
>
> It'd be nice if the license was consistent with Puppet - GPLv2 - but I
> guess not critical.
I would be fine with that and I think it makes sense that it stay
consistent.
greets Pete
On Thursday 17 April 2008, James Turnbull wrote:
> Peter Meier wrote:
> > So after some prework and proposal of DavidS on
> > http://reductivelabs.com/trac/puppet/wiki/CommonModules I would propose
> > to continue with his proposal and try to figure out how we could do some
> > kind of community place for modules which won't get cluttered into
> > x-different module versions which won't be compatible to each another.
>
> This is going to be interesting- some thoughts:
>
> - how do you handle the issue of distribution/platform inter-operability
> in a consistent manner?
After suggestions from the immerda.ch guys, I've created a distro/os aware
branch of my apache module:
http://git.black.co.at/?p=module-apache;a=shortlog;h=distros
The main class dispatches to a specific subclass by $operatingsystem:
class apache {
include "apache::${operatingsystem}"
}
and the additional code can be autoloaded:
apache/manifests/debian.pp contains the apache::debian class:
class apache::debian inherits apache::base {
Package["apache"] { name => apache2 }
# ...
}
This class contains all the Debian (etch) specifica. Since I've no other
systems to test and not enough time anyways, this is rather unfinished, but
the structure and intent should be clear.
> - Also how do you handle cross-module testing, inter-relationships and
> compatibility? You need to consider some form of testing methodology or
> even test scripts to ensure updates to some modules don't break other
> modules.
This was already discussed at length without any definitife outcome. People
seem to do manual testing, report watching and nagios testing.
> > - a central wiki page which would contain all modules inside this
> > community process
> Happy to help with this.
As already mentioned, I started CommonModules and CompleteConfiguration on the
wiki with the intention of kicking off such a community effort, but never
came around to actually do a formal release.
> > - a central git-repository (proposal was to use github.com) with one
> > maintainer for each module.
> Hmm - I think you'd need to consider more than one maintainer - easier
> to ensure a module isn't orphaned. Perhaps a group who look after
> collections of modules.
I think most people will have a single Platform but multiple Applications,
because they come from a "Redhat shop" or - like myself - from a "Debian
shop", but have to run apache, nagios, munin, etc.
This suggests to me that modules (which are mostly per-application) are not
the best fit for maintenance units anyways.
> > - avoid cluttering of modules, the goal is to get one module which
> > should fit for nearly all people. (might be an utopia ;) ) and at the
> > end there shouldn't be in my opinion for example a dozen different
> > modules for apache.
> Maybe a core module and sub-modules by OS?
As detailled above, OS-support within a module can mostly be constrained to a
single autoloaded file in the style of native "providers".
> > - free license: huh let's start another flamewar :P but what i'd like to
> > say is that the modules should be published in the sense of free
> > software (so bsd or gpl) and it should be possible to use them each
> > another. i don't care which license the specific module have, the
> > important thing is that they are free and can be used along each another.
>
> It'd be nice if the license was consistent with Puppet - GPLv2 - but I
> guess not critical.
I released all my modules under a simple BSD license because I don't care. So
GPLv2 would be OK for me too.
Regards, DavidS
- --
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFIDyop/Pp1N6Uzh0URAj5xAJwPT8xZtlEJNkDWz5BOTsS0TAo4PwCfdJof
e51d40DOqRIkQcRuT0UX8mg=
=p0v+
-----END PGP SIGNATURE-----
On Wednesday 23 April 2008, Al @ Lab42 wrote:
> I love your work David but I think that sometimes you overruse
> variables and templates, which leads to some kind of over complexity
> in the modules usage (the user has to understand the module logic and
> know its variables in order to use it).
> Sometimes an user has just to place his configuration file, with
> custom values that don't change much site-wide, on his nodes (read
> "source" rather than "content").
> I wonder if you've considered an elegant way to leave to the user
> (possibly using site-specific subclasses) the decision on how to
> provide configuration files, with templates or sourcing files
> (eventually with different suffixes on the puppetmaster according or
> nodenames or other variables).
While I _do_ tend to overengineer, I believe that this is rather a problem of
documentation than anything else. Additional possibilites for overriding
sources / contents can be added without cost to the basic functioning of the
module like this:
define something(override = '') {
file { blah: }
case $override {
'': {
File[blah] { content => template ( 'blah_default.erb' ) }
}
default: {
File[blah] { source => $override }
}
}
}
Other possibilities include using default places in source arrays or multiple
templates:
file { blah: source =>
[ "puppet:///local/modulename/$fqdn", "puppet:///modulename/default.conf" ] }
See above. Though I'd rather expect this to be implemented on a case-by-case
basis if someone comes up with an actual specific use-case.
Regards, DavidS
- --
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFID6Fi/Pp1N6Uzh0URAjBLAJ9CGMfedIjpEwUg9bug5MeA+pd1HACffHyV
gedKkdBsuzjoN7jO6Km7cCw=
=Cgtt
-----END PGP SIGNATURE-----
On Thursday 15 May 2008, Al @ Lab42 wrote:
> David,
> the discussion about module (and plugins) community standardization is
> not important to let it fade out.
Thanks for picking this up again. I'm currently very much strapped for
time :-(
> Let's start to share sample ideas on how an apache basic module could
> be.
> End of tunnel should be be far :-)
After feedback from Peter, I modified my apache module to start supporting
multiple distros. Basically that means that there is
a) a common "contract" how others can "communicate" with the apache. This is
handled by providing the apache::site and apache::module defines as well as
managed directories and default names, like Service['apache'] and
Exec['reload-apache'].
e.g.:
http://git.black.co.at/?p=module-apache;a=blob;f=manifests/site.pp
b) an 'apache' class that multiplexes to the correct subclass,
basically 'include "apache::${operatingsystem}"' and an apache::base class
defining the primary parts of the module.
http://git.black.co.at/?p=module-apache;a=blob;f=manifests/init.pp
c) operating system specific overrides to handle the diversity.
http://git.black.co.at/?p=module-apache;a=blob;f=manifests/debian.pp
This structure is IMHO very flexible and maintainable, although this specific
instance is not yet split enough (there are still debian bits in ::base).
Regards, DavidS
- --
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFIMnVY/Pp1N6Uzh0URAlo4AJ96ECCHWS04e0NyLt27KbeebYW+3gCbByIG
zT1emSBOWxLPK45g5HV/GOU=
=wn+C
-----END PGP SIGNATURE-----