Announcing Tiny Puppet

[Alessandro Franceschi]

Hi all,

with some hardly hidden pride and excitement I'd like to announce the first release of Tiny Puppet ("Yet Another Puppet Abstraction Layer"), a Puppet module that allows easy, quick and coherent management of virtually any application.

You can read something more about it in this blog post: Introducing Tiny Puppet 

Briefly, a common basic usage pattern, in local site modules and profiles, is as easy as:

tp::install { 'nginx': }

tp::conf { 'nginx':
  template     => 'site/nginx/nginx.conf.erb',
  options_hash => hiera('nginx::options_hash'),
}

and this can be done with ANY application on ANY Operating System (or, to be more precise, to any currently supported application) as long as it can be installed via the native OS packaging system.

The module provides several other defines which, based on the application data, allow quite interesting things, such as an incredibly quick, easy and effortless way to run acceptance tests (or monitoring checks).

For example to run acceptance tests on Centos7 for all the supported application is enough to execute:

bin/test.sh all Centos7 acceptance

this uses the local Vagrant environment to install, test and uninstall applications (read the docs for the prerequisite commands to issue to setup your local environment).

Check here for the current compatibility matrix of different applications on different OS, most of the failures are due to incorrect application data, missing extra repositories (they can be managed directly in the data files), missing default configuration files and other easily fixable issues.

Other defines are available, or planned, for different purposes, they are all based on the applications data which may be easily added and extended to support new OSes:

• tp::install. It installs an application and starts its service, by default
• tp::conf. It allows to manage configuration files
• tp::dir. Manages the content of directories
• tp::stdmod. Manages the installation of an application using StdMod compliant parameters
• tp::line. (TODO) Manages single lines in a configuration file
• tp::repo. (WIP) Manages extra repositories for the supported applications
• tp::concat. (WIP) Manages file fragments of a configuration file
• tp::instance. (TODO) Manages an application instance
• tp::puppi. Puppi integration (Don't worry, fully optional)
• tp::test. Allows quick and easy (acceptance) testing of an application

Tiny Puppet currently requires Ruby > 1.9.x (on the PuppetMaster) and PuppetLabs' stdlib.

It can be plugged without conflicts in any local modules set and can replace or integrate other existing modules: it's all based on defines, you can decide which ones to use for which applications.

Its expected user is the system administrator who knows exactly how to configure his/her applications and needs a standard and quick interface to manage them without the effort to import and study (or even worse, write from scratch) a new module.

I'm looking forward to hear any feedback about it: without false modesties I consider it the result and the essence of years of development and research on modules' reusability and abstraction patterns, but as long as I don't hear others' opinions about it I'll remain with the doubt if it's going to appeal only me :-)

Best,
Alessandro Franceschi

[Felix Frank]

Hi Alessandro,

I did some cursory rummaging in your code and it looks pretty spiffy (in a good way!)

Am I right to assume that most of the module's power is encapsulated in the data/ tree?

This is quite the feat. I'm looking forward to seeing how this will affect the module ecosystem as a whole, as well as common best practices.

Regards,
Felix

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/2455a0cf-96a7-4cd6-8abd-071751f69ebc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alessandro Franceschi]

Hi Felix,

thank you for the feedback, very appreciated.

Yes, most of the (expandability) power is in the data directory, here you can easily add support of a new application or extend the coverage on different OSes. The tp_lookup function fetches it and the defines in the module use it to do various things according to the different use cases (installation, configuration of files or directories and so on).

For example a few hours ago I pushed the first implementation of (automatic) repository management: a few lines of extra data allows the management of a dedicated repo (in this commit https://github.com/example42/puppet-tp/commit/fccaa651197541d38b6d9393ed09b963dadd765b what has been done to install elasticsearch, for example).

There are some little things here and there to manage things in a correct way and make the whole thing work seamlessly, for example an attempt to manage automatically dependencies while allowing users to override them (https://github.com/example42/puppet-tp/blob/master/manifests/conf.pp#L65), the automatic management of repositories, when defined in the data files (https://github.com/example42/puppet-tp/blob/master/manifests/install.pp#L68) or a very easy way to manage acceptance tests (https://github.com/example42/puppet-tp/blob/master/manifests/test.pp#L13),  which in my very naive approach are simply the execution of a check script which may be triggered in whatever way you want (for example in a CI pipeline) ( a bit simpler and quicker to write than a full Beaker test suite, if you ask me).

To cope with application specific configuration settings, sooner or later I'll probably start to add in the data directory default configuration parameters for each application (ie, for apache, DocumentRoot, ServerName and so on) and a sample template that uses this data (merged with the options_hash parameter) to allow users to define as (Hiera) data also application specific configuration settings (it's already possible, actually, but there's no default data set and no sample template).
A prerequisite for this step is probably the need to allow variables interpolation in the data yaml files, as Hiera does (one thing that I still haven't figured out is how to actually use Hiera functions in tp_lookup instead of mimicking its behaviour).

As for the modules ecosystem evolution I have no idea, I know for sure that I'll concentrate myself on writing reusable modules that work at an higher abstraction layer (profiles or stacks, as I call my own approach to such modules) and stop to care about component application modules.  Which means, more or less, that I'm going to throw away (or better, leave their management to volunteer maintainers) 80% of my modules and use, where necessary (since tp doesn't do everything), the best of the existing ones around.

My2c

Alessandro

[jcbollinger]

On Friday, January 2, 2015 3:49:25 AM UTC-6, Alessandro Franceschi wrote:

Hi all,

with some hardly hidden pride and excitement I'd like to announce the first release of Tiny Puppet ("Yet Another Puppet Abstraction Layer"), a Puppet module that allows easy, quick and coherent management of virtually any application.

I remember when you first introduced the Tiny Puppet idea to the group.  I'm amused now that at the time, in the context of the broader discussion, you wrote "Sometimes I wonder if a totally data driven approach is something that is always the right choice."  It doesn't look like TP is totally data-driven, but it sure takes a big leap in that direction.

In any event, TP looks great.  It seems, in part, the natural evolution of the module standards ideas: if one accepts the proposition that most low- (and some not-so-low-)level modules should have standardized form, then not only should we not need multiple modules to manage the same component, we shouldn't really need multiple standardized modules, either.  Why write all that boilerplate?  Automating that sort of thing is what computers are for.

In fact, there are good reasons for modules -- or at least classes -- wrapped around TP resources, but it certainly looks like TP will do a great job of minimizing all the boilerplate, which I guess was a primary objective.

Well done!


John

[Alessandro Franceschi]

On Monday, January 5, 2015 5:57:35 PM UTC+1, jcbollinger wrote:

On Friday, January 2, 2015 3:49:25 AM UTC-6, Alessandro Franceschi wrote:

Hi all,

with some hardly hidden pride and excitement I'd like to announce the first release of Tiny Puppet ("Yet Another Puppet Abstraction Layer"), a Puppet module that allows easy, quick and coherent management of virtually any application.

I remember when you first introduced the Tiny Puppet idea to the group.  I'm amused now that at the time, in the context of the broader discussion, you wrote "Sometimes I wonder if a totally data driven approach is something that is always the right choice."  It doesn't look like TP is totally data-driven, but it sure takes a big leap in that direction.

In any event, TP looks great.  It seems, in part, the natural evolution of the module standards ideas: if one accepts the proposition that most low- (and some not-so-low-)level modules should have standardized form, then not only should we not need multiple modules to manage the same component, we shouldn't really need multiple standardized modules, either.  Why write all that boilerplate?  Automating that sort of thing is what computers are for.

Exactly. Given the difficulties in gathering people around naming standards, which for modules mean basically standard interfaces to their functionalities, and considering the fact that I value the principle of consistency and coherency in a modules ecosystem I considered this approach as my only possibility to achieve it without writing (again) a lot of modules based on a common structure.

It took me some years (I remember a discussion, made several years ago with Teyo Tyree, about the opportunity to reduce all the boilerplate around modules with similar structure) but somehow only now this goal has been achieved.

Honestly this could have happened way before, as there's nothing in Tiny Puppet that could not have been done a few years ago, but I suppose things have their own maturation times.

In fact, there are good reasons for modules -- or at least classes -- wrapped around TP resources, but it certainly looks like TP will do a great job of minimizing all the boilerplate, which I guess was a primary objective.

Well done!

Thank you! Honestly I'm quite proud to listen such words from you :-)

Al