Handling Interdependent Files in a type&provider

[Igor Galić]

Fellow Humans!

i would like solicit your expert knowledge in designing a type & provider for
Apache Traffic Server's [storage.config](http://trafficserver.readthedocs.org/en/latest/reference/configuration/storage.config.en.html)

The basic outline of my general idea, which i implemented with finch's
Filemapper, and finch's help(!) can be found here:

type: https://github.com/Brainsware/puppet-trafficserver/blob/types/lib/puppet/type/trafficserver_storage.rb
provider baseclass: https://github.com/Brainsware/puppet-trafficserver/blob/types/lib/puppet/provider/trafficserver_storage.rb (literally, a class, this was finch's idea ;)
provider for linux: https://github.com/Brainsware/puppet-trafficserver/blob/types/lib/puppet/provider/trafficserver_storage/udev_storage.rb

(ignore it for now;)

now the problem that i'm trying to solve is as follows:

storage.config *drives* the configuration process: when it changes, *something*
on the system changes too. this could either mean a

* directory needs to be created and chowned to the correct user
* udev|devfs entry needs to be made and udev needs to be reloaded

the first case is easy and platform independent (i.e.: it works on all Unix
platforms equally, and traffic server only runs on Unix, so it's fine ;)

the second case however is more problematic as it means we need to keep two
syntactically distinct files in sync. it also means that we need to verify, and
possibly recreate that secondary file if *nothing* in the primary file at all
has changed.

now, from what i gather, a provider cannot be split on a line-by line basis,
but maybe i'm wrong there. in my design above i integrated the directory case -
as an if statement - into the base provider. Considering all the things i've
learned about refactoring in the past three months, this strikes me now as a
profoundly bad move ;)

as for the second case, at least with filemapper, this is really hard to do:
the udev/devfs files would have to be handled in ruby, rather than in
filemapper, if i use inheritance.

and so i'm stuck, and would like some advise on how to handle this properly.

thank you very much in advance.


-- o/~ i
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.g...@brainsware.org
URL: http://brainsware.org/
GPG: 8716 7A9F 989B ABD5 100F 4008 F266 55D6 2998 1641

[Felix Frank]

On 12/05/2014 10:28 AM, Igor Galić wrote:
> * directory needs to be created and chowned to the correct user
> * udev|devfs entry needs to be made and udev needs to be reloaded
>
> the first case is easy and platform independent (i.e.: it works on all Unix
> platforms equally, and traffic server only runs on Unix, so it's fine ;)
>
> the second case however is more problematic as it means we need to keep two
> syntactically distinct files in sync. it also means that we need to verify, and
> possibly recreate that secondary file if *nothing* in the primary file at all
> has changed.

Hi,

I'm not sure that I fully understand a problem, but if at all possible,
I like to try and solve such things by generating additional resources.
The idea is roughly:
* add a generate hook to the trafficserver_storage type
* make it invariably generate a file resource for the directory
* depending on the agent platform, it can
* generate file resources for the udev config
* generate a service resource for the udev refresh

If the user manages any of those in their own manifest, the respective
resource will not be added to the catalog. This might be problematic, of
course. For the service reload, but you might be able to accomodate
using Trevors autonotify hooks.

As for the files, those might fail silently ("fail" only if the manifest
manages a state that is not sufficient for your type's needs;, otherwise
it will Just Work). You might want to raise exceptions if the files are
managed already, so that the user is aware of possible trouble. (You
could even inspect the resources in the catalog and raise very specific
problems that the user can address in their manifest.)

HTH,
Felix

[David Schmitt]

Both parts of the problem strike me as being caused by your try to chain
a secondary configuration off of the storage.config handling. This might
be the most straightforward/logical map of the trafficserver's logic,
but it does not really reflect the reality of puppet managing it. You
seem to have already considered factoring the problem into a puppet
define that manages the storage.config and the directory in parallel
(judging from the comment in [¹]) but I do not quite follow your
reasoning and would think that a separate type to handle the actual
config file and a define to actually configure a tserver::directory,
tserver::raw_device, etc. would allow you to have proper parameters
depending on the configured cache location type and surprisingly also
provide you with a high degree of safety.

> define trafficserver::device($id = undef, $volume = undef) {
> trafficserver_storage { $name: id => $id, volume => $volume; }
> udev::access_rule { $name: user => $trafficserver::user, ... }
> }
>
> define trafficserver::directory($size, $id = undef, $volume = undef) {
> validate_path($name)
> validate_size($size)
> trafficserver_storage { $name: id => $id, volume => $volume, size => $size; }
> file { $name: ensure => directory, force => false, owner => $trafficserver::user, ... }
> }


Splitting it like this would also create independently usable artifacts
(trafficserver_storage parser, udev::access_rule) that allow people to
build their own stuff on top without having to workaround built-in
assumptions. (trafficserver::nbd_mirror anyone?)


Regards, David

[¹]https://github.com/Brainsware/puppet-trafficserver/blob/types/lib/puppet/provider/trafficserver_storage.rb#L102

--
* Always looking for people I can help with awesome projects *
Twitter: @dev_el_ops G+: https://plus.google.com/+DavidSchmitt
Blog: http://club.black.co.at/log/
LinkedIn: http://at.linkedin.com/in/davidschmitt