Puppet Common Modules
Martha Greenberg
project are as follows:
1) To develop a set of standards for writing puppet modules. This is currently
at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards .
2) To develop a set of standards for documenting puppet modules. This is
currently at
http://reductivelabs.com/trac/puppet/wiki/ModuleDocumentationStandards .
3) To develop a standard set of modules that users can just download and use.
These modules will be tested, documented and independent. A RedMine project has
been started for this at http://reductivelabs.com/redmine/projects/show/pcm .
If you would like to get involved with Puppet but are not a Ruby programmer,
here is your chance! We will need help writing documentation, standards and the
modules themselves.
I would like to start the discussion on the standards for writing and
documenting modules both via email and via the wiki. In particular, generating
documentation will require some sort of external tool. We are open to any ideas
on what that tool should be and how the documentation should be structured.
Also, the module writing standards on the wiki are a first draft and suggestions
will be welcomed.
Once the standards are set, everyone will be able to write and submit modules to
this collection and begin using them.
Martha Greenberg
mar...@mit.edu
Randy Bias
strong vote for this tool. For those unfamiliar with TiddlyWiki, it's
essentially a complete Wiki in one page of html using Javascript. The
value here is that it requires no back end, can be sent around like a
normal document, has full Wiki capabilities, and can be checked in as
part of the module code base, including being revisioned alongside of
the code itself.
Randy Bias, chief tactician, neoTactics, Inc.
(877) NEO-TKTX, ran...@neotactics.com
des...@gmail.com
assets.
which modules will be there? will they be small and general or bigger
and more specific? maybe we need a list of modules to start with.
if I write a module or class for each system package, at the end of
the day I have a set of classes with the typical “service ensure
running”, the required “package ensure installed” and maybe some
package configuration files with a “notify the service”. sure this is
useful, but often not enough. for me al least.
but if I start with a class named “my_first_mail_server” something
entirely different happens. the result is a manifest that expresses
all the components and complex interconnections of a very specific
mail-server: the one I want to install and manage.
David Escala
Bryan Kearney
> A new project has been started, called Puppet Common Modules. The goals of this
> project are as follows:
>
> I would like to start the discussion on the standards for writing and
> documenting modules both via email and via the wiki. In particular, generating
> documentation will require some sort of external tool. We are open to any ideas
> on what that tool should be and how the documentation should be structured.
> Also, the module writing standards on the wiki are a first draft and suggestions
> will be welcomed.
I had talked to David Lutterkort about sharing an augeas module [1], so
this seems like a great way to do that and to discuss the common modules
effort as well.
First, on the module. It is built to expose the augeas tool. It accepts
a set of changes as either a single string, or an array of strings. Any
comments on this approach would be appreciated. The goal was to allow a
single puppet task to execute multiple changes as a single unit of work.
This is especially important if one change requires adding several lines
to a file.
Now, on to the common modules. I went through the docs, and tried to
apply them to the augeas module below. The augeas module is really only
a single custom type. However, I like packaging it as a module as this
seems like a good level for re-distribution. The fact that it is a
single custom type may influence the comments below.
Here are some comments I came away with
I) On standards, I have always found that it is difficult to do any sort
of interesting dependencies with items in the form of Foo::Bar. Having
require => [Package[X], Foo::Bar[Y]] always causes me fits. Is there a
documented way to do this, or can we look to make the standard not use ::?
II) For types, should the custom doc be in the type class (as I have it)
or in the mainfest file?
III) I have found that working on many manifests causes me to use the
path to understand what file I am working on. Can we make it the
standard to have the init.pp files import manifests with sensible names?
IV) How should the license information be incorporated with the
documentation? Should it be a separate box, or part of the main doco?
V) Is there any way to pull the documentation from the plugin and use it
to generate the actual documentation? Alot of it is the same information
-- bk
[1] http://git.et.redhat.com/?p=ace.git;a=tree;f=modules/augeas
Al @ Lab42
I'm at disposal for "help writing documentation, standards and the
modules themselves".
Once a definitive standard has been defined I'll for sure convert my
modules set to adhere to it.
Among the various issues of a standardization, I begin with a simple
question for a specific case I've faced.
This is an example, but quite common, I suppose, considering also
David Escala's post:
How to organize in standard modules the configuration of a mailscanner
server?
- Mailscanner can be used with different MTA (postfix, sendmail...).
- It can have some useful addons (mailwatch for mailscanner is a
lovely web interface).
- A working mailscanner module for a complete single box mail server
solution (example: Postfix+MailScanner+Clamav+Spamassassin+SqlGrey
+Mailwatch+Dovecot) requires the configuration of different files of
different services.
How to organize them?
More generally, how a to organize a module for a "major software
package or service" that requires interaction and configuration of
other "major software packages or services"?
And, btw, what's the best place to discuss about the discussion on the
standards?
Best regards,
Al
Evan Hisey
> Among the various issues of a standardization, I begin with a simple
> question for a specific case I've faced.
> This is an example, but quite common, I suppose, considering also
> David Escala's post:
> How to organize in standard modules the configuration of a mailscanner
> server?
> - Mailscanner can be used with different MTA (postfix, sendmail...).
> - It can have some useful addons (mailwatch for mailscanner is a
> lovely web interface).
> - A working mailscanner module for a complete single box mail server
> solution (example: Postfix+MailScanner+Clamav+Spamassassin+SqlGrey
> +Mailwatch+Dovecot) requires the configuration of different files of
> different services.
> How to organize them?
>
> More generally, how a to organize a module for a "major software
> package or service" that requires interaction and configuration of
> other "major software packages or services"?
>
In this vain, a standard approache to variable passing form the either
the site.pp or the node decsriptions to the models should probably be
developed.
Evan
Joshua Timberman
>
> Having recently seen TiddlyWiki in action, I would like to put in a
> strong vote for this tool. For those unfamiliar with TiddlyWiki, it's
> essentially a complete Wiki in one page of html using Javascript.
I think the gist of the 'external documentation' tool is something
like Rdoc, where special comment markup can create nice documentation.
There was discussion about this topic on IRC yesterday.
TiddlyWiki is a great tool (I use it daily), but I don't think it fits
this need.
Adam Jacob
He's got something he's doing for his modules currently that rocks in
this space.
Adam
--
HJK Solutions - We Launch Startups - http://www.hjksolutions.com
Adam Jacob, Senior Partner
T: (206) 508-4759 E: ad...@hjksolutions.com
R.I.Pienaar
----- "Adam Jacob" <ad...@hjksolutions.com> wrote:
> Where's Volcane at?
>
> He's got something he's doing for his modules currently that rocks in
> this space.
Been a tad busy today, but starting to write up a guide to doing it using Natural Docs at http://reductivelabs.com/trac/puppet/wiki/ModuleDocumentationStandards
--
R.I.Pienaar
David Lutterkort
> A new project has been started, called Puppet Common Modules.
This is great news. In a similar vein, the Fedora Infrastructure guys
are trying to come up with guides for best practices for sysadmins for
various situations[1], and I think they intend to back those with puppet
modules. Not sure what (if any) collaboration there makes the most
sense, but probably something to keep an eye on.
> 1) To develop a set of standards for writing puppet modules. This is currently
> at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards .
Some comments:
* '(1) Modules should be independent' ... that seems to be more
based on a limitation of the current tools than something that
is inherently necessary. I think modules fall into two broad
categories: (a) utility modules with general purpose defines
etc. (b) modules that describe some aspect of a system's
configuration (in the sense of Alva Couch's closures) These
should be the defining characteristics of a module, not its
dependencies
* '(2) The standard modules should be downloaded ...' ... is again
more a tools issue; a fixed modulepath is really only helpful if
we have tools to help deploy modules, similar to a package
manager
* '(5) Objects are defined in modules in alphabetical order' Why ?
It would make more sense to organize them in an order that
somehow follows the dependencies between resources.
David
[1] https://fedorahosted.org/csi
Martha Greenberg
I don't think there is any chance a separate file would stay up to date.
Garth Roxburgh-Kidd
will directly remedy my top obstacle in using Puppet and persuading my
team members to do so.
I'll pitch in on the modules, starting with my Python module. For the
time being, I'll whack it up on GitHub.
Yours,
Garth.
Martha Greenberg wrote:
> A new project has been started, called Puppet Common Modules.
Randy Bias
documentation, which is handled well by an RDoc solution. I just
assumed that would be so. I'm more concerned with a non-sysadmin
accessible 'user guide' for people trying to use Puppet modules at a
higher level. But then we're generally trying to make Puppet more
generally accessible. I get that this particular crowd has less use
for a user guide.
To put it a different way, RDoc style == API docs. TiddlyWiki == user
guide docs. A different audience in both cases, but perhaps it's just
something we'll solve separately for CloudScale.
Thanks,
--Randy
Randy Bias, chief tactician, neoTactics, Inc.
(877) NEO-TKTX, ran...@neotactics.com
Mike Pountney
On 30 Jul 2008, at 02:22, Martha Greenberg wrote:
>
> A new project has been started, called Puppet Common Modules. The
> goals of this
> project are as follows:
>
> ...
>
> 3) To develop a standard set of modules that users can just download
> and use.
> These modules will be tested, documented and independent. A RedMine
> project has
> been started for this at http://reductivelabs.com/redmine/projects/show/pcm
> .
>
I think this is a very positive step, and would love to get involved.
Can I make a suggestion that we use a 'common' namespace and module-
name prefix for each of these modules however, to distinguish them
from all the non-standard modules out there? I think most of us have
(eg) an 'ssh' module, with the 'ssh::server' namespace already
defined. I think that using simply 'ssh::server' for the common module
will make migration and integration difficult.
How about we name the modules:
puppet_common_{name} (eg puppet_common_ssh)
[I prefix with puppet_ as this seems to be the standard on github,
though i guess this is not necessary]
and the underlying classes/defines in the modules:
common::{name}::{client,server} (eg common::ssh::server)
This way we can easily just add the modules into our manifest trees
and start using and inheriting them as we see fit, for example:
class ssh::service inherits common::ssh:server {
file {"/etc/ssh/sshd_config":
content("ssh/my_sshd_config_template")
}
}
Hope this makes sense,
Mike
David Schmitt
> On Wed, 2008-07-30 at 04:22 +0300, Martha Greenberg wrote:
>> A new project has been started, called Puppet Common Modules.
>
Yeah, really! A big "Thanks!" shout out to Martha for getting the ball
rolling! You can count me definitely in and it would be great for me to
donate my modules and help on further working on them (although I do
have to admit that I have less time to spend recently).
>> 1) To develop a set of standards for writing puppet modules. This is currently
>> at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards .
>
> Some comments:
>
> * '(1) Modules should be independent' ... that seems to be more
> based on a limitation of the current tools than something that
> is inherently necessary. I think modules fall into two broad
> categories: (a) utility modules with general purpose defines
> etc. (b) modules that describe some aspect of a system's
> configuration (in the sense of Alva Couch's closures) These
> should be the defining characteristics of a module, not its
> dependencies
> * '(2) The standard modules should be downloaded ...' ... is again
> more a tools issue; a fixed modulepath is really only helpful if
> we have tools to help deploy modules, similar to a package
> manager
> * '(5) Objects are defined in modules in alphabetical order' Why ?
> It would make more sense to organize them in an order that
> somehow follows the dependencies between resources.
Exactly what I wanted to say. Additionally:
1) If two modules need the same (ruby) function, there is no way that
should work[1] except to move the function to a shared module (or add a
dependency from one module to the other).
[1] that also means that pluginsync should probably fail if there are
two definitions for the same function.
5) I can't write how bad an idea I think this is.
Later today I'll take a shot at polishing my common module with the new
Doc Standards and restructuring it a bit for a more "modern" manifest
style. We can see how it progresses from there.
I'll be on IRC the next hours ...
Regards, DavidS
David Schmitt
> I think this is a very positive step, and would love to get involved.
> Can I make a suggestion that we use a 'common' namespace and module-
> name prefix for each of these modules however, to distinguish them
> from all the non-standard modules out there? I think most of us have
> (eg) an 'ssh' module, with the 'ssh::server' namespace already
> defined. I think that using simply 'ssh::server' for the common module
> will make migration and integration difficult.
>
> How about we name the modules:
>
> puppet_common_{name} (eg puppet_common_ssh)
>
> [I prefix with puppet_ as this seems to be the standard on github,
> though i guess this is not necessary]
>
> and the underlying classes/defines in the modules:
>
> common::{name}::{client,server} (eg common::ssh::server)
>
> This way we can easily just add the modules into our manifest trees
> and start using and inheriting them as we see fit, for example:
>
> class ssh::service inherits common::ssh:server {
> file {"/etc/ssh/sshd_config":
> content("ssh/my_sshd_config_template")
> }
> }
I'm not convinced that the common:: namespace prefix would be a good
idea. Quite to the contrary, I would recommend that users put their
local changes into the namespace _below_ the PCMs:
| class ssh::server::extra_secure inherits ssh::server {
| # ...
| }
On the point of migration, I think the namespace clashing is the lesser
problem. The fear of change in established systems is a much greater
hindrance. Having a central "blessed" repository from reductivelabs will
help here in the long run much more than hiding stuff in another namespace.
See also Bryan Kerney's comment I about not liking the "require =>
Foo::Bar[]" stuff. While I can't say I totally support his point, I'm
definitely in favour of less namespace nesting.
Regards, DavidS
David Schmitt
Jeroen van Meeuwen (GMail) schrieb:
> David Lutterkort wrote:
>> On Wed, 2008-07-30 at 04:22 +0300, Martha Greenberg wrote:
>>> A new project has been started, called Puppet Common Modules.
>> This is great news. In a similar vein, the Fedora Infrastructure guys
>> are trying to come up with guides for best practices for sysadmins for
>> various situations[1], and I think they intend to back those with puppet
>> modules. Not sure what (if any) collaboration there makes the most
>> sense, but probably something to keep an eye on.
>>
>
> As one of the authors on the ConfigurationManagement document (which is
> a work in progress still), and initiator of puppetmanaged.org, I will
> initially be using the modules available there -which doesn't mean no
> collaboration can take place of course.
>
> puppetmanaged.org has a very specific target though, and that is
> providing modules that do not need to be changed downstream in order to
> start using them, yet still providing the alternative locations of
> files, modules and manifests for (multiple-) domain specific environments.
>
> I hope this makes sense but if it doesn't let me know and I'll try to
> elaborate some more.
I do think that any "Common" repository will have to have modules which
do not need to be locally patched to make them work in the local
environment.
Surely, using git and stuff makes local patches easier, but this should
be mostly work that can be pushed "upstream".
> I'll certainly be lurking on this list and PRM and DavidS' manifests and
> other interesting places.
See you there ;)
Regards, DavidS
Mike Pountney
On 1 Aug 2008, at 11:25, David Schmitt wrote:
>> ...
>> This way we can easily just add the modules into our manifest trees
>> and start using and inheriting them as we see fit, for example:
>>
>> class ssh::service inherits common::ssh:server {
>> file {"/etc/ssh/sshd_config":
>> content("ssh/my_sshd_config_template")
>> }
>> }
>
> I'm not convinced that the common:: namespace prefix would be a good
> idea. Quite to the contrary, I would recommend that users put their
> local changes into the namespace _below_ the PCMs:
>
> | class ssh::server::extra_secure inherits ssh::server {
> | # ...
> | }
>
>
But this will then force us to alter the PCMs for our own
environments, making it difficult to maintain upstream changes.
Using a common:: prefix (or common_ for the module directory) means
that we do not need to change the PCMs for our own environments, where
naturally template files and such are going to be more variable - we
can override the 'golden' PCM modules as we see fit in our own modules.
I guess my point is that I see the PCMs as a useful base for our own
modules - base *::server and *::client classes, useful defines,
plugins, parser functions, that will help us easily create our own
modules for our more tailored environments. I don't think it is
possible, certainly with the more complicated subsystems, to provide a
PCM module that will handle everyone's needs directly. We will have to
alter them someway, and I think by extension into our own (non common)
modules is the cleanest way of preserving the sanctity of the PCMs
themselves.
> On the point of migration, I think the namespace clashing is the
> lesser
> problem. The fear of change in established systems is a much greater
> hindrance.
I'm not sure what you mean here. Replacing my existing ssh module with
an entirely new one that I have to understand prior to deployment, is
not a task I relish, and ssh is quite a simple example. I'd rather
install the PCMs alongside my existing modules and gradually shift
over to using them: not through fear, but to minimise risk.
> Having a central "blessed" repository from reductivelabs will
> help here in the long run much more than hiding stuff in another
> namespace.
>
This is what i would love to see - a stable, well tested, base module
system. With this, we can make sensible assumptions about other
peoples manifest trees, and use the PCM module functions, classes, et
al as we see fit.
Regards,
Mike
Jeroen van Meeuwen (GMail)
>
> On 1 Aug 2008, at 11:25, David Schmitt wrote:
>
>>> ...
>>> This way we can easily just add the modules into our manifest trees
>>> and start using and inheriting them as we see fit, for example:
>>>
>>> class ssh::service inherits common::ssh:server {
>>> file {"/etc/ssh/sshd_config":
>>> content("ssh/my_sshd_config_template")
>>> }
>>> }
>> I'm not convinced that the common:: namespace prefix would be a good
>> idea. Quite to the contrary, I would recommend that users put their
>> local changes into the namespace _below_ the PCMs:
>>
>> | class ssh::server::extra_secure inherits ssh::server {
>> | # ...
>> | }
>>
>>
>
> But this will then force us to alter the PCMs for our own
> environments, making it difficult to maintain upstream changes.
>
Actually what I think it forces you to do -nevermind the advocacy thing
going on here-, is to think about how you can change the changes you
make into changes that can go upstream, submit them as a proposal
regardless of whether you implement them in your own environment
straight-up, and then follow-up with upstream to see what might further
be needed to improve adoption of your changes in the upstream module, if
anything, guarding that your motivation for changing your modules in the
first place isn't being neglected as your changes move into a acceptable
changeset to apply to the upstream module, and then rebasing when
upstream merges in your changes.
This (and here's the advocacy I was talking about) ensures you that;
- your changes are long-term sustainable as upstream takes into account
changes made upstream are tested for all use-cases covered in
current-release-1-minor, or migrate applicable use-cases into
current-release+1-major before letting the general public put it in
production stage (*uch*, insert grain of salt here)
- you do not have to test, re-test, rebase, merge back in, the changes
you apply to version X in order to be able to update to version Y, which
saves you headaches, frustration, and in the end, a lot of time (and
thus money, which is were you get your customer or employer to
appreciate investing a comparatively little amount of time in getting
your changes accepted by upstream)
- you can go away from your customer or employer without having to
instruct them to hire another expensive puppet master (no pun intended),
because it's actually upstream maintaining the modules they want to use
and apply to their environment which at some point is a core-business
impacting piece of infrastructure and such pieces glue together with
using puppet and using standardized modules. All you need to tell your
customer or employer is to hire someone who can spell "puppet" right.
Kind regards,
Jeroen van Meeuwen
-kanarip
Garth Roxburgh-Kidd
a a common set of modules, and that they'd have short and obvious
names. I'd have been shocked to discover I had to include common::ssh
— or, worse, require => Common::something["name"]. Or is it
common::Something["name"]?
I'd prefer to rename my modules to custom_* or companyname_*, keep the
PCM modules unprefixed, and make life easier for newcomers to Puppet.
They'll outnumber us in the medium to long term.
(In a later post, you refer to the difficulty of replacing your ssh
module with the common ssh module. You shouldn't. Rename your ssh
module, instead. We're treating configurations as code, after all, so
why not refactor it?)
Yours,
Garth.
Garth Roxburgh-Kidd
private implementations of common modules unless absolutely necessary.
The whole idea with Puppet, after all, is to not have to re-invent the
automation wheel every time you change jobs.
If I needed features lacking in the common module, I'd fork from it
rather than rewrite, and submit patches to hopefully close the gap.
Yours,
Garth.
2008/8/2 Jeroen van Meeuwen (GMail) <kan...@gmail.com>:
…
Jeroen van Meeuwen (GMail)
> Jeroen, this is a key point: as administrators, we want to avoid
> private implementations of common modules unless absolutely necessary.
> The whole idea with Puppet, after all, is to not have to re-invent the
> automation wheel every time you change jobs.
>
> If I needed features lacking in the common module, I'd fork from it
> rather than rewrite, and submit patches to hopefully close the gap.
>
Well, that is your choice and I respect that, as I think will everyone
else on this planet, but I hope you can appreciate where the statements
I've made are, to some extend, valuable for you and anyone else to
consider, maybe on a case-by-case basis, regardless of whether you
choose to go by them or ignore them.
Jeroen van Meeuwen (GMail)
> Jeroen, this is a key point: as administrators, we want to avoid
> private implementations of common modules unless absolutely necessary.
> The whole idea with Puppet, after all, is to not have to re-invent the
> automation wheel every time you change jobs.
>
> If I needed features lacking in the common module, I'd fork from it
> rather than rewrite, and submit patches to hopefully close the gap.
>
Hmm... reading your message a second time I think I may have
misunderstood you the first time I read it. My bad if that's the case,
I'm sorry, and thank you.
Kind regards,
Jeroen "not a native english speaker" van Meeuwen
-kanarip
Garth Roxburgh-Kidd
a decent common module, it's odd to write your own. If the common
module lacks something you need, you should minimize your changes and
submit them as a patch.
The Python community has a term: "Pythonic". The language is absurdly
clean. If your code isn't clean, there's probably something wrong.
This is doubly true for libraries. If using your library makes my code
look odd, there's probably something wrong with the design of your
library.
Similarly: Puppet minimizes — almost eliminates — the effort of
controlling the configuration of large numbers of hosts. If you find
yourself spending as much time hand-coding your Puppet modules as you
use to spend administering machines, there's probably something wrong.
Sadly, "puppetic" sounds awful.
David Schmitt
Hash: SHA1
On Saturday 02 August 2008, Mike Pountney wrote:
> On 1 Aug 2008, at 11:25, David Schmitt wrote:
> >> ...
> >> This way we can easily just add the modules into our manifest trees
> >> and start using and inheriting them as we see fit, for example:
> >>
> >> class ssh::service inherits common::ssh:server {
> >> file {"/etc/ssh/sshd_config":
> >> content("ssh/my_sshd_config_template")
> >> }
> >> }
> >
> > I'm not convinced that the common:: namespace prefix would be a good
> > idea. Quite to the contrary, I would recommend that users put their
> >
> > local changes into the namespace _below_ the PCMs:
> > | class ssh::server::extra_secure inherits ssh::server {
> > | # ...
> > | }
>
> But this will then force us to alter the PCMs for our own
> environments, making it difficult to maintain upstream changes.
>
> Using a common:: prefix (or common_ for the module directory) means
> that we do not need to change the PCMs for our own environments, where
> naturally template files and such are going to be more variable - we
> can override the 'golden' PCM modules as we see fit in our own modules.
This is a strawman. Nobody needs to touch anything within the ssh module to
create a ssh::server::extra_secure class. Of course, autoloading won't help
you with this design, but that's no big deal for local classes and surely
less a problem than getting autoload to work around the common:: prefix.
> > On the point of migration, I think the namespace clashing is the
> > lesser
> > problem. The fear of change in established systems is a much greater
> > hindrance.
>
> I'm not sure what you mean here. Replacing my existing ssh module with
> an entirely new one that I have to understand prior to deployment, is
> not a task I relish, and ssh is quite a simple example. I'd rather
> install the PCMs alongside my existing modules and gradually shift
> over to using them: not through fear, but to minimise risk.
One can do a quite risk-less global rename of the local ssh module to
ssh_local to avoid having to replace it immediately. Either way won't spare
you the neccessary migration and testing. But using a common:: prefix breaks
autoloading and will be a millstone around everyones neck forever.
Regards, DavidS
- --
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iD8DBQFIlGHi/Pp1N6Uzh0URAjseAKCDw9LfSxQhhk7gD7NSYOVwVF8mzACfeIIh
Es4SXBn3hXTYpiHvWbhUKpI=
=wNdW
-----END PGP SIGNATURE-----
James Turnbull
Hash: SHA1
David Schmitt wrote:
> One can do a quite risk-less global rename of the local ssh module to
> ssh_local to avoid having to replace it immediately. Either way won't spare
> you the neccessary migration and testing. But using a common:: prefix breaks
> autoloading and will be a millstone around everyones neck forever.
+1.
Regards
James
- --
Author of:
* Pulling Strings with Puppet
(http://www.amazon.com/gp/product/1590599780/)
* Pro Nagios 2.0
(http://www.amazon.com/gp/product/1590596099/)
* Hardening Linux
(http://www.amazon.com/gp/product/1590594444/)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFIlJDD9hTGvAxC30ARAqxlAJ9AMy+A5QkeoqgVvyLhAod/EFmXwgCfUKJA
2yzAjkgT6LPeql1UEBwaaB4=
=prdp
-----END PGP SIGNATURE-----
Mike Pountney
(*) Firstly, after reading the responses to my post, I recognise that
my 'common::' idea ('scuse the pun) is not sensible. Stop reading now
if you like :)
Hopefully this discussion has raised some interesting points though
(which have taught me some new puppet techniques, woo!), hence the
comments to the comments below:
On 2 Aug 2008, at 14:32, David Schmitt wrote:
>>
>> Using a common:: prefix (or common_ for the module directory) means
>> that we do not need to change the PCMs for our own environments,
>> where
>> naturally template files and such are going to be more variable - we
>> can override the 'golden' PCM modules as we see fit in our own
>> modules.
>
> This is a strawman. Nobody needs to touch anything within the ssh
> module to
> create a ssh::server::extra_secure class. Of course, autoloading
> won't help
> you with this design, but that's no big deal for local classes and
> surely
> less a problem than getting autoload to work around the common::
> prefix.
>
Good point, I'd never considered the fact that the namespaces and
module 'packaging' are separate things, (with the exception of
autoloading, as you mention below).
>
> One can do a quite risk-less global rename of the local ssh module to
> ssh_local to avoid having to replace it immediately. Either way
> won't spare
> you the neccessary migration and testing.
In my particular example, it's also going to involve me renaming many
of my classes and defines. It's a reasonable job, but not a huge task
I admit.
This is a minor quibble compared to the benefit new users get from PCM
modules though. See (*) ;)
> But using a common:: prefix breaks
> autoloading and will be a millstone around everyones neck forever.
>
I see what you mean. This would force the use of 'puppet_common_ssh::server'
as the module name (based on my suggestion of using
puppet_common_ssh as the module name), which is just horrid. See (*)
My main reasoning behind this discussion was that the PCM modules are
likely to be fairly immutable, yet will need to be altered by some to
fit into their (eg) company policy or existing environments. Templates
in particular are likely going to need to be changed by people, in
ways that are not necessarily going to be useful to upstream.
Puppet does (as you show above) provide many ways to allow us to
enhance/alter the functionality of them locally, without requiring
changing the PCM modules themselves. This makes me happy, as it solves
my problem.
I guess my point then is, in the PCM Best Practices, how would we
recommend that this is done?
Options so far seem to be (forgive my rough notes):
*) Extend the PCM namespace, as per David's
ssh::server::secure_config, in a separate module (my_ssh)
- maintains the namespace hierarchy
- breaks autoloading
*) Create a new top-level namespace, eg my_ssh::server, in my_ssh module
- avoids namespace clashes with the PCM module
- breaks the namespace hierarchy (eg auto-tagging, will need to do
puppetd -v -o --tags 'my_ssh,ssh' rather than just --tags 'ssh')
*) Fork/Branch the PCM module
- changes are easy to make, no new modules to create.
- requires branch management/merging to keep PCM module up-to-date.
- best solution for changes that are expected to move upstream.
Regards,
Mike
David Schmitt
Hash: SHA1
On Saturday 02 August 2008, Mike Pountney wrote:
> (*) Firstly, after reading the responses to my post, I recognise that
> my 'common::' idea ('scuse the pun) is not sensible. Stop reading now
> if you like :)
Not at all, people who learn are much more interesting to read, than people
who don't!
> Options so far seem to be (forgive my rough notes):
>
> *) Extend the PCM namespace, as per David's
> ssh::server::secure_config, in a separate module (my_ssh)
> - maintains the namespace hierarchy
> - breaks autoloading
>
> *) Create a new top-level namespace, eg my_ssh::server, in my_ssh module
> - avoids namespace clashes with the PCM module
> - breaks the namespace hierarchy (eg auto-tagging, will need to do
> puppetd -v -o --tags 'my_ssh,ssh' rather than just --tags 'ssh')
>
> *) Fork/Branch the PCM module
> - changes are easy to make, no new modules to create.
> - requires branch management/merging to keep PCM module up-to-date.
> - best solution for changes that are expected to move upstream.
Don't forget that local classes do not _have_ to be in modules. A
simple "import 'local_changes/*.pp'" can help much here. Therefore I think
the first option would be the best.
Regards, DavidS
- --
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iD8DBQFIlYJn/Pp1N6Uzh0URAiDzAJ9+XjkSnnqKQvF1zTbAoEw+0QhkdgCeJ+vW
dR128zPe/WPhqSkj+qsRfwk=
=8cPp
-----END PGP SIGNATURE-----
Martha Greenberg
here:
The truth is, that PCM is really three projects:
1) Coming up with a set of standards for the "right" way to write a common
module, so it can be used by as many people as possible. In particular, setting
clear ways to handle as many OSs as possible, and getting the modules to be
stand-alone. I would like Puppet beginners to be able to download starter
modules and have them just work.
The modules that are out right now are great. However, they are split into lots
of different repositories, they often only work on specific OSs, and they
contain lots of site-specific code. This means they are not as useful as they
could be. Most of the time, a user cannot use them as-is.
Here's what I'd like the new Puppet user experience to be:
1) download puppet
2) check out/download PCM
3) modulepath=/etc/puppet/modules:/etc/puppet/pcm-modules
4) user set up nodes with pcm modules.
5) oh wait, you need a different sshd config file? set this up:
class mysite::ssh-server inherits ssh::server {
File["sshd_config"] source => "puppet:///mysite/sshd_config"
}
or class myssh:server inherits ssh::server {}
2) Devising a way to document modules. Ideally, coming up with a tool that
could create stub docs from puppet code. This should probably be split out into
a separate project.
3) Getting current and new module developers under one roof and taking the
existing modules and standardizing and documenting them.
I don't think we should be starting from scratch. But right now, we have no way
of coordinating this work, or letting other people contribute.
Also, I would like to reduce the amount of duplication. Most likely, everyone
on this list has written an SSH module. 90% of us could use the same module, if
it was done right. Just like Puppet abstracts out a lot of repetitive sysadmin
work, I'd like PCM to take that a step further...
Coordinating and organizing these projects is certainly much hard than the
technical aspects. I'm willing to work on this, but we need lots of help. The
announcement was intended more as a call for volunteers and a way to start the
discussion than "This is how things are going to be".
Martha
PS - On a another note, I see this project as a good way to give people who are
not Ruby programmers a way to contribute to Puppet.
Jeroen van Meeuwen (GMail)
> I'm resending this to puppet-users, since a lot of same questions are coming up
> here:
>
>
> The truth is, that PCM is really three projects:
>
> 1) Coming up with a set of standards for the "right" way to write a common
> module, so it can be used by as many people as possible. In particular, setting
> clear ways to handle as many OSs as possible, and getting the modules to be
> stand-alone. I would like Puppet beginners to be able to download starter
> modules and have them just work.
>
A concern here on my part is that if the modules are required to be
entirely standalone, a lot of duplication (in different namespaces) is
going to happen.
Just one example is managing puppet (and the puppetmaster) with a puppet
module, that enables one to have puppet pull updated module sources from
an upstream SCM (branching included, so that MultipleEnvironments are
properly facilitated). Right now this requires one to also have the
module for the preferred SCM in use. Although that might not be a
priority for PCM right now, I would like to take into consideration to
not require the modules to be stand-alone.
Then again, maybe, because of the stand-alone requirement, such a puppet
module should not require the SCM module and not pull from upstream
automagically, until you set $pull_from_scm = 1 (and only then require
the SCM module). This we should take into very careful consideration as
I think having common puppet modules is one thing, and then having a
million settings you need to go through to get it to do anything is another.
One of the other things I think common puppet modules are also great
for, is to enable a sort of de-facto standard in common system
administration, although again that might not at all be a priority at
this point.
> 2) Devising a way to document modules. Ideally, coming up with a tool that
> could create stub docs from puppet code. This should probably be split out into
> a separate project.
>
I'm afraid the documentation needed for some of the modules out there
(and I don't mean the ssh module everyone has), needs to be so extensive
as to explain use-cases in a clear and understandable way that it'll be
obfuscating the code just a little too much. I think little code vs.
lots of comments/documentation preferably does not come from the same file.
Just some thoughts ;-)
Tim Stoop
I'm not very active on the mailinglist, but...
On 3 aug, 16:15, Martha Greenberg <mart...@mit.edu> wrote:
> PS - On a another note, I see this project as a good way to give people who are
> not Ruby programmers a way to contribute to Puppet.
spend some time on this. Especially since we plan to release our
internal modules soon. (David asked me to voice my support, so
consider this my vote of confidence! ;-))
--
Kind regards,
Tim
Digant C Kasundra
<mar...@mit.edu> wrote:
>
> 1) To develop a set of standards for writing puppet modules. This is
> currently at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards .
I'm not sure I like the ::common naming scheme. To me, ssh::server implies
a class that handles the most common case and the less common would inherit
from that and go in a namespace below that (ssh::server::weird). To me,
this more closely resembles an "object-oriented" model.
Digant C Kasundra
<da...@schmitt.edv-bus.at> wrote:
>| class ssh::server::extra_secure inherits ssh::server {
>| # ...
>| }
>
>
> On the point of migration, I think the namespace clashing is the lesser
> problem. The fear of change in established systems is a much greater
> hindrance. Having a central "blessed" repository from reductivelabs will
> help here in the long run much more than hiding stuff in another
> namespace.
I can see where this could be a problem if one PCM module were to depend on
another. There would either have to be a standard practice where PCM gets
the top level or the local institution reserves the top level. Mixing and
matching would be a problem because eventually namespaces would definitely
collide and if a PCM module ssh::blah relied on PCM module tivoli::blah but
an institution already have a tivoli::blah then something has to give.
Maybe everything should go under pcm:: namespace? This is what you sort of
do with things like URNs or to a lesser degree OIDs in LDAP.
Digant C Kasundra
<ja...@lovedthanlost.net> wrote:
>> One can do a quite risk-less global rename of the local ssh module to
>> ssh_local to avoid having to replace it immediately. Either way won't
>> spare you the neccessary migration and testing. But using a common::
>> prefix breaks autoloading and will be a millstone around everyones neck
>> forever.
>
> +1.
Ah, right, I forgot about the autoloading. Locally, we actually have 3
module paths. One for just regular modules, and one for clients and one
for services (as in IT services, not system level services) and to make
autoloading work, we've had to prefix the actual module names with c_ or
s_, which is less then ideal.
Having one big common module or pcm module would not be good. It would be
nice to say that a particular namespace actually correlates to a module
path. I smell a feature request.
Jeroen van Meeuwen (GMail)
I'll take ssh::server as an example...
What is it you can customize to ssh::server, other then the source the
sshd_config is coming from, whether you harvest and manage the host keys
and then... what else?
Then, what do you really need to customize and what part of that
customization is it you really need to manage from the manifest? Putting
the right files in the right place is one thing, putting the right
content in the right files in the right place is another.
Here's an example of a setup that runs in several organizations. Like
always, one needs a little different customization then the other.
The sshd_config file is pulled from (in order of appearance):
puppet:///ssh/sshd_config.$hostname
puppet:///ssh/sshd_config
Any customization for a small number of hosts goes directly into a
sshd_config.$hostname file, whereas customizations per group-of-hosts
goes into sshd_config.group, having symlinks from sshd_config.$hostname
to sshd_config.group. Another example:
sshd_config.bofh-only only allows members of group bofh (a little evil
administrators group) to log on via ssh.
Servers that need this customization have a sshd_config.$hostname
symlink to the sshd_config.bofh-only file. Eg:
sshd_config.ldap01 -> sshd_config.bofh-only
sshd_config.ldap02 -> sshd_config.bofh-only
sshd_config.xen01 -> sshd_config.bofh-only
sshd_config.xen02 -> sshd_config.bofh-only
(etc, etc)
Given this mechanism, whether you use $hostname or $fqdn, anyone can do
any type of customization to the sshd_config file as long as they put
the file (or link) in the right place with the right name.
We all know that once we start templating sshd_config there is no end to
it, and we could better find a way to parse the sshd_config man-page
into a puppet manifest, forcing the "user" (who is actually an
administrator) to walk through numerous settings before being able to
start using the module.
Now, am I missing something here? How far can we take this relatively
simple mechanism? I already know how far in case of the ssh module, and
some other modules, but I'm asking you... Would a simple mechanism like
that serve your use-case, given a change here and there?
For reference, I've also included a link to the actual module running
this show[1]. It takes into account seperate source trees for domain
name spaces (private/$domain), organizations (files/) and module
defaults (ssh/).
I hope you get the point, and I'd love to hear your answers,
Kind regards,
Jeroen van Meeuwen
-kanarip
[1]
http://git.puppetmanaged.org/?p=modules/ssh/.git;a=blob;f=manifests/init.pp
Digant C Kasundra
server and I prefer to have the different explicit in the class name rather
than automagic file loading. The reason is simple: when I use a tool or
just list classes.txt and browse the classes applied to a system, I want to
be able to tell right there what the system is doing or should be doing.
For instance, if I can't connect to a server from an off campus IP, I want
to be able to look at the classes and see if perhaps I'm not using the
ssh::global class, which doesn't restrict ssh connections to just campus
IPs.
So, yes, sshd_config could be handled just by setting up the manifests to
look for .$hostname and then fall back to a more general config but then I
have to dig through my manifests to see what behaviors I have setup a
particular manifests to do instead of knowing from just the name.
Plus, not all things are just the config changes. For instance, the
example I use of ssh version ssh::global is actually related to the
iptables rules that get dropped in and not anything to do with sshd
configuration.
And I'll reiterate just now nice it is to examine the classes being applied
to a server to catch things like a server using ssh::rootenabled instead of
interpreting the manifests or examining all the files in the files dir to
see if a server is using that. That ability to rely on the class names to
peer into that sort of thing is going to be critical for our "one portal"
setup that will allow us to examine all facts about a system from one page.
--On Tuesday, August 05, 2008 12:51:21 AM +0200 "Jeroen van Meeuwen
Jeroen van Meeuwen (GMail)
> To use our modules as an example, we change several things about our ssh
> server and I prefer to have the different explicit in the class name rather
> than automagic file loading. The reason is simple: when I use a tool or
> just list classes.txt and browse the classes applied to a system, I want to
> be able to tell right there what the system is doing or should be doing.
> For instance, if I can't connect to a server from an off campus IP, I want
> to be able to look at the classes and see if perhaps I'm not using the
> ssh::global class, which doesn't restrict ssh connections to just campus
> IPs.
>
> So, yes, sshd_config could be handled just by setting up the manifests to
> look for .$hostname and then fall back to a more general config but then I
> have to dig through my manifests to see what behaviors I have setup a
> particular manifests to do instead of knowing from just the name.
>
> Plus, not all things are just the config changes. For instance, the
> example I use of ssh version ssh::global is actually related to the
> iptables rules that get dropped in and not anything to do with sshd
> configuration.
>
> And I'll reiterate just now nice it is to examine the classes being applied
> to a server to catch things like a server using ssh::rootenabled instead of
> interpreting the manifests or examining all the files in the files dir to
> see if a server is using that. That ability to rely on the class names to
> peer into that sort of thing is going to be critical for our "one portal"
> setup that will allow us to examine all facts about a system from one page.
>
Thank you for the details on how you think what I set out in my previous
email wouldn't work for you! If I understand you correctly, and please
do correct me if I'm wrong, you're saying that having separate
(sub-)classes for separate ssh setups is a matter of convenience to you,
over having multiple sources, am I right? This led me to conclude it is
not so much a technical issue, hence that's why I'm asking.
Thomas Bellman
> I'm just wondering... humor me. I mean this in the best possible way.
>
> I'll take ssh::server as an example...
>
> What is it you can customize to ssh::server, other then the source the
> sshd_config is coming from, whether you harvest and manage the host keys
> and then... what else?
On the nodes in the clusters i manage using Puppet, I do the following:
* Copy /etc/ssh/ssh_host_{rsa,dsa}_key{,.pub} from a central location
(four files; but each node has its own set of four files).
* Make sure that "HostbasedAuthentication" is set to "yes" in
/etc/ssh/sshd_config.
* Make sure that "IgnoreRhosts" is set to "no" in /etc/ssh/sshd_config.
* Install /etc/ssh/shosts.equiv from a template file.
* Install /root/.shosts from the same template file as above.
Note especially that I do *not* manage the entire /etc/ssh/sshd_config,
just a couple of individual settings. I believe my OS vendor is better
than me on knowing what is sensible and safe values for most of those
settings. I don't want to spend time tracking the changes they make
between updates of the openssh-server package, between OS releases, or
between OS distributions. And other sites are likely to want to change
other settings than I do.
Similarly, for the ssh client I do:
* Install /etc/ssh/ssh_known_hosts from a template (that reads the various
host keys from a central location).
* Ensure that the settings
HostbasedAuthentication yes
EnableSSHKeysign yes
exists in the "Host *" section in /etc/ssh/ssh_config.
> Putting
> the right files in the right place is one thing, putting the right
> content in the right files in the right place is another.
On another site I run multiple instances of the ssh server, with
different config files, on the same machine. So it's not always to
easy to even know *where* to put the config files...
/Thomas Bellman
David Schmitt
In my reading there is only one reason a module contains a ::common
class, and that is "sharing resources between multiple user-visible
classes and defines."
In #6 of the Standards(-to-be) I used it for example to split out the
os-independent part of the manifest. As can be seen there, the real
user-visible class is still "ssh::server", which multiplexes according
to $operatingsystem into the right "implementation".
I guess I should make that more clear, or did I misunderstand you
completely?
Regards, DavidS
Digant C Kasundra
(GMail)" <kan...@gmail.com> wrote:
> Thank you for the details on how you think what I set out in my previous
> email wouldn't work for you! If I understand you correctly, and please
> do correct me if I'm wrong, you're saying that having separate
> (sub-)classes for separate ssh setups is a matter of convenience to you,
> over having multiple sources, am I right? This led me to conclude it is
> not so much a technical issue, hence that's why I'm asking.
It is both a technical and convenience issue. The technical side is that
we don't just configure the ssh server in that class but also the
associated firewall rules, which are different for different classes
depending on where we want to allow connections from. Having multiple
sources doesn't quite address that.
Digant C Kasundra
<da...@schmitt.edv-bus.at> wrote:
>>> 1) To develop a set of standards for writing puppet modules. This is
>>> currently at http://reductivelabs.com/trac/puppet/wiki/ModuleStandards
>>> .
>>
>> I'm not sure I like the ::common naming scheme. To me, ssh::server
>> implies a class that handles the most common case and the less common
>> would inherit from that and go in a namespace below that
>> (ssh::server::weird). To me, this more closely resembles an
>> "object-oriented" model.
>>
>
> In my reading there is only one reason a module contains a ::common
> class, and that is "sharing resources between multiple user-visible
> classes and defines."
>
> In #6 of the Standards(-to-be) I used it for example to split out the
> os-independent part of the manifest. As can be seen there, the real
> user-visible class is still "ssh::server", which multiplexes according
> to $operatingsystem into the right "implementation".
>
> I guess I should make that more clear, or did I misunderstand you
> completely?
So, ssh::server includes ssh::server::common and ssh::server::debian? That
definitely seems reverse of what I would consider an inheritance model
approach. What doesn't the os-independant stuff go into ssh::server and
ssh::server::debian inherit from that? Maybe we're both trying to say the
same thing but are talking around each other. Am I still misunderstanding?
James Turnbull
Hash: SHA1
Digant C Kasundra wrote:
>> I guess I should make that more clear, or did I misunderstand you
>> completely?
>
> So, ssh::server includes ssh::server::common and ssh::server::debian? That
> definitely seems reverse of what I would consider an inheritance model
> approach. What doesn't the os-independant stuff go into ssh::server and
> ssh::server::debian inherit from that? Maybe we're both trying to say the
> same thing but are talking around each other. Am I still misunderstanding?
- From my reading you're both saying the same thing.
Regards
James Turnbull
- --
Author of:
* Pulling Strings with Puppet
(http://www.amazon.com/gp/product/1590599780/)
* Pro Nagios 2.0
(http://www.amazon.com/gp/product/1590596099/)
* Hardening Linux
(http://www.amazon.com/gp/product/1590594444/)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFImLlB9hTGvAxC30ARAt/uAKCevMDPD4cCuhct2OLr0xJwTBg+oACdGdDa
sFVr7qNOsGw3qokfff1344E=
=78H/
-----END PGP SIGNATURE-----
David Schmitt
to the ruby core]
Digant C Kasundra schrieb:
Yes, I think I failed to explain the difference between actual
"user-visible" parts and the "under-the-hood" parts which make up the
mechanics.
To copy my (slightly enhanced) example from wiki:ModuleStandards :
| class ssh::server::common {
| service { 'ssh::server': ... }
| }
| class ssh::server::debian inherits ssh::server::common {
| Service['ssh::server'] { name => 'ssh' }
| }
| class ssh::server {
| include "ssh::server::$operatingsystem"
| }
This structure works by splitting the different responsibilities into
separate parts:
1) "Type": A common basic structure for all: The ssh::server::common
class defines all major parts of a ssh server. In this simplified
example, only the Service['ssh::server'] is described. A higher quality
implementation could provide a placeholder for the package and default
users for privilege separation as well as the basic relationships
between this resources.
2) "Provider": OS-specific implementation of details. In this case,
there is only one implementation: the class ssh::server::debian. There
the actual name of the init script is provided. A higher quality
implementation would provide hasstatus/hasrestart and so on, fill out
other details and add release-specific overrides.
3) "Resource": Easy way to use for the common user. Just "include
ssh::server" will bring in all the goodness for enabling a ssh server on
this node.
This structure has the following properties:
*) fail-save: a missing implementation for a OS leads to a compile
error; having two implementations for a OS (e.g. new module version and
user provided one) will lead to a obvious error too.
*) common interface: every implementation will provide a
"Service['ssh::server']" because it is specified in the ::common class.
This enables the user to blindly notify=>Service['ssh::server'] (or
whatever else is needed/available) without having to think about the
specific OS' details underneath.
Also the ::common class leaves out all the gritty and changing details,
like release differences and such thing. This makes the base class much
more legible and helps to distinguish the relevant structure from the
implementation details.
*) support for a yet unsupported OS is easily added: add a new file with
ssh::server::new-os. As soon as this class gets loaded, puppet will use
it where applicable. no need to touch the module at all for this and the
file can be sent as-is for inclusion in the module.
*) general extensibility:
class ssh::server::weird inherits ssh::server {
# include OS-dependent weirdness here
include "ssh::server::${operatingsystem}::weird"
# Add OS-independent weirdness
ssh::conf { "SshBanner": value => "very weird server 2.3"; }
}
Again, this has the same fail-save properties as the original structure,
since it won't compile on a unsupported OS, and the same pattern can be
applied to add another layer again.
This pattern can also be used to add OS-specific extensions.
Regards, DavidS
Trevor Vaughan
Basically, you're looking at a heck of a lot of string creep with some
of these suggestions.
I've been trying to be good and properly nest things but some users of
my results get a bit tired of mis-typing things like:
ssh::server::debian::slightmod::evenslightermod
And tend to prefer:
ssh::evenslightermod which uses 'inherits' to mask everything.
It's not as elegant, but it's way more typeable.
I'm not sure what the actual solution is, but it seems that this way
lies madness if not.....*gasp*....GUIs!
T
Evan Hisey
In this same vain, it really looks like we are trying to do to much
work to amek teh common modules to general. No matter how flexible we
try and make them they will _never_ be right for most use cases. I
thkn the best route here is to make the whole PCM manifest work
cleanly in itself and set up good sound default system setting that
can be easily customized _in_ the module. this way some one can take a
complete generic configuration of say a web server, ssh, and a
hardening module, to create a local puppet manifest an d have all the
parts play nice together. From there they can then customize or grow
teh solution out to meet the specific site needs.
After all , we really should not be assuming this system is for use
by dummies. When it comes to server and infrastructure maintenance
issues, anyone in charge of that should really take the time to learn
how the tools work.
Evan
The Anarcat
> I've been trying to be good and properly nest things but some users of
> my results get a bit tired of mis-typing things like:
> ssh::server::debian::slightmod::evenslightermod
>
> And tend to prefer:
> ssh::evenslightermod which uses 'inherits' to mask everything.
I agree. In fact, here we use koumbit::ssh::server. ie. we *prefix* our
namespace in front of the regular namespace for our customizations.
But I think that ssh::server::debian does make some sense and is
important as you'll also need ssh::client::debian and therefore can't
shortcut to ssh::debian (although there might be some commonality
between "servers" and "clients" that might be factored out in
ssh::debian...)
A.
--
Information is not knowledge
Knowledge is not wisdom
Wisdom is not truth
- Frank Zappa
Nigel Kersten
> In this same vain, it really looks like we are trying to do to much
> work to amek teh common modules to general. No matter how flexible we
> try and make them they will _never_ be right for most use cases. I
> thkn the best route here is to make the whole PCM manifest work
> cleanly in itself and set up good sound default system setting that
> can be easily customized _in_ the module. this way some one can take a
> complete generic configuration of say a web server, ssh, and a
> hardening module, to create a local puppet manifest an d have all the
> parts play nice together. From there they can then customize or grow
> teh solution out to meet the specific site needs.
>
> After all , we really should not be assuming this system is for use
> by dummies. When it comes to server and infrastructure maintenance
> issues, anyone in charge of that should really take the time to learn
> how the tools work.
Evan has nicely summarized the disquiet I've felt throughout this thread.
--
Nigel Kersten
Systems Administrator
Tech Lead - MacOps
James Turnbull
Hash: SHA1
I agree totally with Evan/Nigel that all sys admins should understand
how the module works - no one should blindly apply configuration. I
don't think anyone sees PCM as doing that though. PCM is like a
framework - a skeleton that demonstrates capability and best practice.
But I think the nature of the modules we produce could significantly
impact how easy it is for a new Puppet user to understand those modules.
~ In this spirit we should produce generic modules - and expect
customisation to happen. My principal concern is to what definition of
"generic" should modules adhere? To a single OS/platform? To a
"conceptual" model? To multiple platforms - Linux, BSD, OSX?
I'd like to see a generic module - for example apache - that specifies
all the required base configuration and demonstrates/supports multiple
platforms using sub-classes. Probably not all platforms but broadly
speaking I think Debian/RH/*BSD~OSX are a good baseline. Maybe Solaris
too for some/all modules.
This approach isn't going to meet everyone's needs - not possible - but
it will:
a) Demonstrate the basics
b) Demonstrate the cross-platform nature of Puppet - a critical concept
c) Hopefully be easily extensible/customisable.
There is a question (a mostly technical/style one) of HOW that
customisation should be achieved and articulated - namespaces,
inheritance, overrides, etc, etc. That's just a technical issue to
resolve though and I think some good discussion is going on here.
I personally don't overly care how we do it and if the agreed result is
something like apache::server::debian and people simply modify the
existing module rather than override I'd be happy.
Regards
James Turnbull
- --
Author of:
* Pulling Strings with Puppet
(http://www.amazon.com/gp/product/1590599780/)
* Pro Nagios 2.0
(http://www.amazon.com/gp/product/1590596099/)
* Hardening Linux
(http://www.amazon.com/gp/product/1590594444/)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFImjrF9hTGvAxC30ARAi8wAKC4hjJ8/7OYpA24NsQBdcUWA9MIqQCeJkJA
T7Axprf6iE4gFtEigGmWblk=
=Qu9O
-----END PGP SIGNATURE-----
David Schmitt
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Nigel Kersten wrote:
> | On Wed, Aug 6, 2008 at 1:03 PM, Evan Hisey <ehi...@gmail.com> wrote:
> |
> |> In this same vain, it really looks like we are trying to do to much
> |> work to amek teh common modules to general. No matter how flexible we
> |> try and make them they will _never_ be right for most use cases. I
Every apache admin will need apache::conf_snippet and a significant
portion will need apache::virtual_host.
> |> thkn the best route here is to make the whole PCM manifest work
> |> cleanly in itself and set up good sound default system setting that
> |> can be easily customized _in_ the module. this way some one can take a
> |> complete generic configuration of say a web server, ssh, and a
> |> hardening module, to create a local puppet manifest an d have all the
> |> parts play nice together. From there they can then customize or grow
> |> teh solution out to meet the specific site needs.
> |>
> |> After all , we really should not be assuming this system is for use
> |> by dummies. When it comes to server and infrastructure maintenance
> |> issues, anyone in charge of that should really take the time to learn
> |> how the tools work.
> |
> | Evan has nicely summarized the disquiet I've felt throughout this thread.
> |
>
> I agree totally with Evan/Nigel that all sys admins should understand
> how the module works - no one should blindly apply configuration. I
> don't think anyone sees PCM as doing that though. PCM is like a
> framework - a skeleton that demonstrates capability and best practice.
>
> But I think the nature of the modules we produce could significantly
> impact how easy it is for a new Puppet user to understand those modules.
> ~ In this spirit we should produce generic modules - and expect
> customisation to happen. My principal concern is to what definition of
> "generic" should modules adhere? To a single OS/platform? To a
> "conceptual" model? To multiple platforms - Linux, BSD, OSX?
Conceptual model, like "ssh means OpenSSH, and Service[ssh::server] is
running" (which, coincidentally is what ssh::server::common would
describe, following my example from yesterday evening.
> I'd like to see a generic module - for example apache - that specifies
> all the required base configuration and demonstrates/supports multiple
> platforms using sub-classes. Probably not all platforms but broadly
> speaking I think Debian/RH/*BSD~OSX are a good baseline. Maybe Solaris
> too for some/all modules.
I'm all for a big group-hugging collaboration with people responsible
for apps (single module, concept, prototype on a single os) or platforms
(single os implementation in multiple modules) to support as much
platforms as puppet can do.
> This approach isn't going to meet everyone's needs - not possible - but
> it will:
>
> a) Demonstrate the basics
> b) Demonstrate the cross-platform nature of Puppet - a critical concept
> c) Hopefully be easily extensible/customisable.
>
> There is a question (a mostly technical/style one) of HOW that
> customisation should be achieved and articulated - namespaces,
> inheritance, overrides, etc, etc. That's just a technical issue to
> resolve though and I think some good discussion is going on here.
I'm totally with you there.
Regards, DavidS
Digant C Kasundra
<peiri...@gmail.com> wrote:
We've avoided this many subclasses. There is something to be said for not
breaking every little thing into different classes. For instance, do you
have lots of scenarios where you have ssh server but not the client or
visa-versa? We don't, so we just have an ssh module that handles both
server and client, thus eliminating one level of subclassing.
Digant C Kasundra
<ja...@lovedthanlost.net> wrote:
> PCM is like a
> framework - a skeleton that demonstrates capability and best practice.
I'm not sure how much people will agree on a best practice for some of the
things PCM will model. Even with configuring and running puppet, something
we all on this list have in common, there is much disagreement on what is
"best." But it will certainly be a framework for people that want to take
it for one way to do things that will work and if nothing else, as you
stated, it will serve as a good starting ground for someone trying to
tackle writing manifests for a particular purpose.
Digant C Kasundra
<da...@schmitt.edv-bus.at> wrote:
> Yes, I think I failed to explain the difference between actual
> "user-visible" parts and the "under-the-hood" parts which make up the
> mechanics.
>
> To copy my (slightly enhanced) example from wiki:ModuleStandards :
>
>| class ssh::server::common {
>| service { 'ssh::server': ... }
>| }
>| class ssh::server::debian inherits ssh::server::common {
>| Service['ssh::server'] { name => 'ssh' }
>| }
>| class ssh::server {
>| include "ssh::server::$operatingsystem"
>| }
Very novel. I probably need to think a bit more about how this would work
in a practical sense.
I can imagine it getting a bit annoying if you have a lot of subclasses and
have to therefore have a lot of ::$operatingsystem:: namespaces. I would
say for something simple, it would still be okay to just put the os
specific difference in ssh::server.
For instance, in our ssh (our module handles both client and server b/c we
never install one without the other so no real need to break apart), we use
a case statement off of operatingsystem to determine what the package name
is. Other than that, everything is the same. So, not sure if I would want
to break that out into ssh::debian and ssh::redhat. But in the case where
there are lot of differences, for legibility alone, it is a good idea to
follow the structure you've outlined.
David Schmitt
> --On Wednesday, August 06, 2008 04:38:37 PM +0200 David Schmitt
> <da...@schmitt.edv-bus.at> wrote:
>
>> Yes, I think I failed to explain the difference between actual
>> "user-visible" parts and the "under-the-hood" parts which make up the
>> mechanics.
>>
>> To copy my (slightly enhanced) example from wiki:ModuleStandards :
>>
>> | class ssh::server::common {
>> | service { 'ssh::server': ... }
>> | }
>> | class ssh::server::debian inherits ssh::server::common {
>> | Service['ssh::server'] { name => 'ssh' }
>> | }
>> | class ssh::server {
>> | include "ssh::server::$operatingsystem"
>> | }
>
> Very novel. I probably need to think a bit more about how this would work
> in a practical sense.
>
> I can imagine it getting a bit annoying if you have a lot of subclasses and
> have to therefore have a lot of ::$operatingsystem:: namespaces. I would
> say for something simple, it would still be okay to just put the os
> specific difference in ssh::server.
Definitely. On the up side, that can be much more compact, on the
downside it is harder to extend locally (patch instead of added file).
> For instance, in our ssh (our module handles both client and server b/c we
> never install one without the other so no real need to break apart), we use
> a case statement off of operatingsystem to determine what the package name
> is. Other than that, everything is the same. So, not sure if I would want
> to break that out into ssh::debian and ssh::redhat. But in the case where
> there are lot of differences, for legibility alone, it is a good idea to
> follow the structure you've outlined.
If you look at my actual ssh module at [1], you see that my ssh::server
actually inherits from ssh::client and I do handle a bit more than
package+service --- specifically fixing the user and group that are
created for priviledge separation and stuff. I don't know whether other
distros might not differ more on this and on which files to repair
ownership afterwards.
I also don't know yet how to adapt the structure as it is currently to
my outlined method. I presume I will fold the classes into one and
provide a override to disable all server functionality.
Regards, DavidS
[1] http://git.black.co.at/?p=module-ssh;a=blob;f=manifests/init.pp;hb=HEAD
Trevor Vaughan
many cases as possible.
We have the $operatingsystem and related facter variables and all of
the types that I've used try to do something sane given an OS (or
other fact about the system).
So, it would make sense to me that modules follow the same general concept.
It seems that a general rule for a given module would be:
class foo {
case $operatingsystem (or whatever, it would be really nice to be
able to OR or AND these) {
os1: { do os1 stuff },
os2: { do os2 stuff },
default: { if there is something sane for everyone, do it,
otherwise throw an error as running a module on an unsupported OS }
}
}
Unfortunately, I've got a few cases that look like the following:
class foo {
case $operatingsystem {
os1: {
case $operatingsystemversion {
2: { do os1.2 stuff },
3: { do os1.3 stuff}
},
os2: {
case $operatingsystemversion {
8: { do os 2.8 stuff},
12: { do os 2.12 stuff}
},
etc...
}
}
And, as you can see, that's really ugly but absolutely functional.
T
Garth Roxburgh-Kidd
> Personally, I think that the operating system should be hidden in as
> many cases as possible.
+1
Francois Deppierraz
> And tend to prefer:
> ssh::evenslightermod which uses 'inherits' to mask everything.
You can always create a specific module for your company which contains
customizations to publically availables modules.
class mycompany::ssh inherits ssh::server::foo::bar
François
Trevor Vaughan
That's definitely true, and not a bad idea at all.
It looks like I'm going to have to finally put together that puppet
manifest parser to graphviz parser after all. All of the inheritance
is getting too confusing without a map.
Does anyone know if there are any hooks into the current manifest
parser, or would it be an inline mod?
I thought about parsing the YAML, but it doesn't seem to keep the
inheritance metadata. . .
Trevor
Digant C Kasundra
<peiri...@gmail.com> wrote:
> Unfortunately, I've got a few cases that look like the following:
>
> class foo {
> case $operatingsystem {
> os1: {
> case $operatingsystemversion {
> 2: { do os1.2 stuff },
> 3: { do os1.3 stuff}
> },
> os2: {
> case $operatingsystemversion {
> 8: { do os 2.8 stuff},
> 12: { do os 2.12 stuff}
> },
> etc...
> }
> }
Me too. What was proposed was to break the stuff into foo::debian so you
could just do:
class foo {
include foo::$operatingsystem
}
class foo::os1 {
case $operatingsystemversion {
2: { do os1.2 stuff },
3: { do os1.3 stuff}
}
}
All you get is legability so you don't have one big honking class with a
tone of case statements.
Paul Lathrop
but check genconfig and you'll find it.
Sent from Gmail for mobile | mobile.google.com
Trevor Vaughan
I thought that was post-compile, not pre-compile. I'll try it and see
what I get.
Thanks,
Trevor