Feature proposal: "metadata" metaparameter

[Reid Vandewiele]

Hey puppet-dev,

Working with several large Puppet deployments I've repeatedly seen an interest in improving the ability to link Puppet management information back to internal business processes. Towards that end I've written a proposal to add a language feature, a non-operative metaparameter, to allow attaching user-defined data to resources for later reporting purposes.

The feature proposal below is placed here for discussion, feedback, and reactions to the proposal and suggested solution.

Consider the following use cases

As a Puppet code workflow designer,
I want to my teams to create document-as-you-go annotations in their code for:

• Business-process identifiers pertaining to configuration items
• Human-readable descriptions of configuration items
• Responsible party for configuration items

So that later when Puppet is is making changes to a system, I can easily look up who, what, and why.

As a Puppet content developer,

I want to attach ownership information to resources,

So my team can be identified and alerted if Puppet has a problem applying my resources.

As a change deployment approver,

When reviewing changes made in a lower-tier environment to decide if it's ok to promote,

I want to consume a high-level summation for the meaning of changes applied to a system,

So I can quickly get a sense of what's being done,

Without needing to read low-level Puppet implementation code.

As a customer receiving a Puppet-managed server as a service, 

I want to review a human-readable list of all “stuff” managed by Puppet on my system,

So I can have some understanding of what Puppet is doing.

Currently, we don't have a way to attach any kind of self-documenting information to Puppet content, or produce dynamic reports about what Puppet is doing on a target system based on documentation or annotations written for human consumption.

Further, while we have puppet-strings to create self-documenting code, we don't have any tooling to create self-documenting catalogs or reports—these being examples of artifact outputs produced by the potentially documented code.

Feature Request

Provide a solve for use case #1 above, in support of the rest of the use cases.

Suggested Solution

A simple way to solve for this would be to add a new, non-operative metaparam, "metadata". As a metaparam, "metadata" would be available to specify on any Puppet resource or class. Because it is non-operative and for reporting purposes only, the addition of the metaparam would by itself constitute foundational delivery of this feature. Metaparameters are present in catalogs and queryable from PuppetDB for building reports already; the only thing missing is a hook to hang this sort of data on to begin with.

The metadata metaparam could be built on by later PE features to produce polished reports, but would not depend on such a thing to deliver value today.

Example usage:

notify { 'example':
  message  => 'this is a notify resource',
  metadata => 'this is some metadata, a description, about it',
}

file { '/tmp/example.conf':
  ensure   => file,
  owner    => 'root',
  metadata => {
    'description' => 'Metadata does not need to be a string',
    'product'     => 'Example Product',
    'owner'       => 'Kelly',
  },
}

What about tags?

Tags have two characteristics that make them unsuitable to fully address these use cases.

1. Tags cannot contain text written for human consumption, such as descriptions or text-form comments, due to not allowing spaces and other special characters.
2. Tags propagate. Especially for long-form comments, it may be desirable NOT to propagate them, as duplication and accidental scope escape can be undesirable.

For some use cases the propagation feature of tags may make them more desirable than using metadata. Metadata would be different enough that a meaningful decision could be made about which to use for a given purpose.

Compatibility considerations

As far as I can tell, there are no concerns with regards to backwards compatibility. If an existing class or type already uses a parameter called "metadata", that will mask/override a metaparam by the same name.

Probable Implementation Difficulty: trivial

Improper diff to demonstrate:

diff --git a/lib/puppet/type.rb b/lib/puppet/type.rb

index 8951854ddd..5f230e3cea 100644

--- a/lib/puppet/type.rb

+++ b/lib/puppet/type.rb

@@ -1714,6 +1714,16 @@ class Type

+  newmetaparam(:metadata) do

+    desc "User-supplied descriptive or administrative information about

+      this resource. Metadata is non-operative, but will be visible in the

+      node's catalog and can be queried for reporting purposes."

+  end

+

 

Naming considerations

Based on the Wikipedia definition of the term, metadata seems to describe exactly the generalized purpose of attaching data to Puppet resources non-operatively, and fit the use cases given above. 

https://en.wikipedia.org/wiki/Metadata

The term "annotation" was considered as well, based on looking at similar constructs in Kubernetes. The definitions of both of those nouns more or less matches the use case for the proposed feature, but the natural use and rich explanations for "metadata" are a clear and obvious match to the use case, whereas the rich explanations for "annotation" are less so.

https://en.wikipedia.org/wiki/Annotation

~ fin ~

The corresponding ticket for this proposal is https://tickets.puppetlabs.com/browse/PUP-10491.

Initial discussion is best suited for the mailing list. This thread is now open for feedback and discussion. After any initial discussion, updates will be made to the ticket.

-- 

Reid Vandewiele

Puppet Solutions Architect