Module Standards discussion
Blake Barnett
up before in various forms, now though, I think enough people are
attempting to write shareable code that this is becoming an urgent
requirement for the community to take advantage of this effort.
Coming up with standards for this will be crucial to the success of
sharing any of this stuff.
I believe we need something akin to WSDL[1], I'm NOT saying should use
XML to describe these, something more like YAML[2] or JSON[3] would
probably be more in line. Modules can easily become extremely
complex, especially when definitions, native types, and plugins are in
the mix. I think this can be very ad-hoc. A tool that can parse and
notify about naming conflicts, and missing dependencies would be
crucial. This can grow into a fuller feature-set if it's warranted.
It would also be cool to have a module skeleton generation tool that
fits best practices.
A trivial example could look something like this:
module_postfix.yml
---
name: postfix
version: 0.1
description: Yay.... postfix.
provides:
classes: postfix, postfix::ldap, postfix::gerbils
definitions:
- name: postfix_spamq
description: Specifies blah de blah
required_parameters:
- name: ensure
- parameter_options: present, absent
default: present
optional_parameters:
native_types: postfixregex
dependencies:
native_types:
parser_functions:
modules:
facter_plugins:
Obviously this could easily get out of hand. It may be tempting to
add external dependency tracking for things like gems and files,
packages, etc. I think we just need to focus on internal puppet
requirements. Perhaps pre-parsing the module and providing
information about failures would be sufficient to show what external
dependencies haven't been satisfied.
Along the same lines, I think we also need a standard way of embedding
documentation in manifests so that something like rdoc is easily doable.
I'm willing to write code, develop the specs, etc. I'd love to hear
what everyone else thinks.
-Blake
[1] http://en.wikipedia.org/wiki/Web_Services_Description_Language
[2] http://en.wikipedia.org/wiki/YAML
[3] http://en.wikipedia.org/wiki/JSON
Andrew Shafer
Blake,
I started thinking about this before and I don't believe WSDL is the right metaphor, but some standard is definitely in order.
What we really need at some point is package management for modules.
I started down the yaml road, drawing inspiration from the deb packaging, with a control.yaml file describing each module. My first pass shares some similarities with your proposed prototype. (I stopped working on it because priorities got in the way and because I started thinking it was worth separating the modeling of environments to simplify the module logic first.)
On the topic of code sharing, I think it is unrealistic to expected every piece of puppet code will run perfectly on every platform, furthermore module developers actually testing a module against every platform is even less realistic. This should not be a barrier to sharing.
I propose that modules are 'certified' against a given OS version, similar to the deb architecture field, and possibly application versions as well. To what extent this can be or needs to be encoded in the control file is open for discussion, but there needs to be a convention to make it explicit and obvious to a person getting the module and potentially the system as well.
At least, if we make 'platform' explicit and documented on a module, everyone has a starting point for their platform (and sometimes is will even 'just work'). Maybe eventually we have a more sophisticated infrastructure and convention for blessing the modules, until then it will basically be a circle of trust.
Cheers,
Andrew
Blake Barnett
>
> Blake,
>
> I started thinking about this before and I don't believe WSDL is the
> right metaphor, but some standard is definitely in order.
Yeah, I was shooting for something between package management and API/
interface details.
>
> What we really need at some point is package management for modules.
>
> I started down the yaml road, drawing inspiration from the deb
> packaging, with a control.yaml file describing each module. My
> first pass shares some similarities with your proposed prototype. (I
> stopped working on it because priorities got in the way and because
> I started thinking it was worth separating the modeling of
> environments to simplify the module logic first.)
Could you explain a little more what you mean by modeling envinroments?
>
> On the topic of code sharing, I think it is unrealistic to expected
> every piece of puppet code will run perfectly on every platform,
> furthermore module developers actually testing a module against
> every platform is even less realistic. This should not be a barrier
> to sharing.
>
> I propose that modules are 'certified' against a given OS version,
> similar to the deb architecture field, and possibly application
> versions as well. To what extent this can be or needs to be encoded
> in the control file is open for discussion, but there needs to be a
> convention to make it explicit and obvious to a person getting the
> module and potentially the system as well.
>
> At least, if we make 'platform' explicit and documented on a module,
> everyone has a starting point for their platform (and sometimes is
> will even 'just work'). Maybe eventually we have a more
> sophisticated infrastructure and convention for blessing the
> modules, until then it will basically be a circle of trust.
I think this platform vetting/certifying belongs on a module
repository site. It should be easy for users to report success and
failure for a module (or any other piece of sharable code such as
native functions, etc.) on different platforms and distributions. It
makes a lot of sense to make that very visible in the same place
people would download the code (I also think such a site should have a
CLI as a primary UI). I'm sure people will still end up reporting
bugs in bug trackers anyway, but I'm sure Luke would appreciate
reducing that as much as possible. :)
-Blake
Peter Meier
> On the topic of code sharing, I think it is unrealistic to expected every
> piece of puppet code will run perfectly on every platform, furthermore
> module developers actually testing a module against every platform is even
> less realistic. This should not be a barrier to sharing.
>
> I propose that modules are 'certified' against a given OS version, similar
> to the deb architecture field, and possibly application versions as well.
> To what extent this can be or needs to be encoded in the control file is
> open for discussion, but there needs to be a convention to make it explicit
> and obvious to a person getting the module and potentially the system as
> well.
I like this idea and it was kind of the way that I thought to get
started with a community driven module plattform.
I think the module development is always a more pragmatic and
step-by-step ahead job than saying from the beginning on: it have to
work on every plattform.
And for me it's also out of question that a maintainer should be develop
them on any plattform. For me her or his job is rather to merge patches
from people using other plattforms together, try to keep the common
guideline of documentation etc.
So I really like you proposal of certification and think that's the
pragmatic way we should look for.
greete Pete
Randy Bias
My challenges right now aren't really around packaging a module. At
the end of the day that's not so hard. It's more around surfacing
information about how to use a module externally and putting modules
together.
Granted, our use case isn't typical, but to be honest I think the true
power of puppet won't be seen until we can share and mix modules. I
like certifications, but I also like having both some kind of RDoc
facility and a mechanism for have another tool to be able to
understand the interfaces to modules.
I actually think we need both long term. I'm less interested in
packages. My inclination is to leave modules in git and simply build
tools that can mix, merge, and package modules on demand.
--Randy
Randy Bias, chief tactician, neoTactics, Inc.
(877) NEO-TKTX, ran...@neotactics.com
On Apr 19, 2008, at 7:51 AM, Andrew Shafer wrote: