Custom / nested resource types - crackpot ideas

[JonY]

Is it (reasonably) feasible to add new resource types? I don't want to disassemble the whole code base but if there is a known path to this I'd like to read about it.

Some thoughts:It strikes me that there are at a few situations that I run into with this SW that would make for decent additions. 

1. "once": do a given resource type once and set a fact to not repeat it. I keep kludging solutions to this. 

2. "log": set a log level and location for a resource / node / etc. This would help to figure out "WTF is going on".

3. "return value": set a fact with the failure code for a given operation or agent run. I keep finding cases where the agent is failing and the server isn't logging it. It isn't until I log in and run the agent that I see the error(s).

4. "counter": set a fact with the # of times a given operation has been performed on a system. This would help to track down systems that are thrashing because of competing operations.

These could either be additional params for existing resources or an outer layer to same.

[jcbollinger]

On Wednesday, October 29, 2014 8:48:56 AM UTC-5, JonY wrote:

Is it (reasonably) feasible to add new resource types? I don't want to disassemble the whole code base but if there is a known path to this I'd like to read about it.

Yes.

1. Defined types are resource types.  If you have a Puppet manifest set of any significant size then you are probably using these already.
2. You can write your own plugin / native / Ruby resource types, too.  There is official documentation for how to do this, and there are plenty of examples -- many in Puppet's own code, and some in publicly available modules.
   

Some thoughts:It strikes me that there are at a few situations that I run into with this SW that would make for decent additions. 

1. "once": do a given resource type once and set a fact to not repeat it. I keep kludging solutions to this. 

I could imagine a defined type wrapper that would help you do that.

 

2. "log": set a log level and location for a resource / node / etc. This would help to figure out "WTF is going on".

Likewise here, more or less, but I'm inclined to think that it would be easier to just use a collector to override selected resources' loglevels.  Nodes don't have loglevels, though, so you can't do it at that scope by any means.

 

3. "return value": set a fact with the failure code for a given operation or agent run. I keep finding cases where the agent is failing and the server isn't logging it. It isn't until I log in and run the agent that I see the error(s).

You could write custom facts that extract failure codes for selected prior operations from the system logs.  That doesn't require a resource.  Resources don't have a sense of a "return code", however, nor any consistent hooks into the operations performed by their providers, so a generic mechanism to capture and record such information directly from resources does not sound feasible to me.

 

4. "counter": set a fact with the # of times a given operation has been performed on a system. This would help to track down systems that are thrashing because of competing operations.

Puppet already provides per-run reports describing which resources were modified, if you configure it to do so.  It does not have much of a generic sense of finer-grained "operations".

 

These could either be additional params for existing resources or an outer layer to same.

If you are inclined to use a custom-modified version of Puppet then you can do pretty much anything.  Some things you suggest doing will require a lot more work than others, however, and you may run into issues with third-party modules (unless you hack those, too).  Puppet already provides alternatives for most of the things you described, and if it were me, I would not entertain the possibility of hacking up Puppet itself.


John

[Felix Frank]

Hi,

+1 to John, you're thinking of some wheels to reinvent.

On 10/29/2014 02:48 PM, JonY wrote:
> Is it (reasonably) feasible to add new resource types? I don't want to
> disassemble the whole code base but if there is a known path to this
> I'd like to read about it.
>
>
> Some thoughts:It strikes me that there are at a few situations that I
> run into with this SW that would make for decent additions.
>
> 1. "once": do a given resource type once and set a fact to not repeat
> it. I keep kludging solutions to this.

Best practice is to make all resource idempotent, so that it's fine for
Puppet to continuously check sync states.

If you really need things to run once only, you might get away cheaply
with a "has puppet ever run" fact, so that you can have a whole class of
things that will only ever run after provisioning. (Sure, if you ever
add to that class, the new resources won't run on old agents...)

> 2. "log": set a log level and location for a resource / node / etc.
> This would help to figure out "WTF is going on".

I don't think I follow. You want this information on every single agent
run? Why?

Do you not add the --debug flag whenever something is fishy, and can
safely ignore the gory details again once things are fixed?

> 3. "return value": set a fact with the failure code for a given
> operation or agent run. I keep finding cases where the agent is
> failing and the server isn't logging it. It isn't until I log in and
> run the agent that I see the error(s).

Again, don't really follow. Have you considered gathering and
summarizing reports? If you set up PuppetDB and Puppet Explorer /
PuppetBoard, you can very quickly determine which agents are generating
errors, both constantly and transiently.

> 4. "counter": set a fact with the # of times a given operation has
> been performed on a system. This would help to track down systems that
> are thrashing because of competing operations.

And once again, this is what PuppetDB will just do for you.

> These could either be additional params for existing resources or an
> outer layer to same.

I don't think that you can add metaparameters through modules. That
would indeed require patches to the Puppet core.

Meta-Types (like the resources type) to perform some extra operations on
other resources from the catalog - that would be a different story. You
could likely cobble something up in a module.

Cheers,
Felix