Sharing Puppet Functions

[William Leese]

Hi,

The accepted standard for sharing code seems to be modules, which doesn't seem a nice fit if you simply want to share a single custom Puppet function.

Having an easy way to look for Puppet functions would likely greatly help the development of such, providing a nice set of examples and avoiding duplicate work. 

Currently I simply create a module for each function and upload to the forge. Whats your approach? One monolithic module? Would that be easily findable on the Forge (tagging won't scale)? etc. etc.

[jcbollinger]

On Thursday, April 24, 2014 2:45:21 AM UTC-5, William Leese wrote:

Hi,

The accepted standard for sharing code seems to be modules, which doesn't seem a nice fit if you simply want to share a single custom Puppet function.

Why not?

 

Having an easy way to look for Puppet functions would likely greatly help the development of such, providing a nice set of examples and avoiding duplicate work. 

I don't follow.  If you don't like modules for this purpose, then what would be the characteristics of your ideal solution?

 

Currently I simply create a module for each function and upload to the forge. Whats your approach? One monolithic module? Would that be easily findable on the Forge (tagging won't scale)? etc. etc.

Custom functions are valuable and widely used, but they are not a focal feature of Puppet.  It is uncommon for people to provide or seek a single custom function in isolation.  Instead, people usually are after a broader facility.  Typically that consists of classes and/or resource types (native or defined), possibly with associated custom functions.  In other words, a module.

If you are writing custom functions so general that they are not associated with any particular system configuration regime, then it might be suitable to collect several into a library module.  PuppetLabs's 'stdlib' module follows this route (though it does provide a resource type or two along with the many functions).


John

[Chris Johnson]

Hi.

Had the following error in my messages log from the puppet gent daemon,
i.e. the client side:

{/Stage{main}/web_server/File{/var/www/html/index.html]) Could not
evaluate: Could not retrieve file metadata for
puppet://modules/web_services/client1/html/index.html: Connection
refused - connect(2)

Now maybe I wouldn't be quite so annoyed if I hadn't been chasing file
protocols and network problems for four days on this one. And the
computer gods know I've google it enough and seen all kinds of responses.

Please, please, please would it be too much trouble for a simple "file
or directory not found" error instead? That's what the problem was. But
I just spend four days googling this and try to find nonexistent network
problems among other things.

I like the idea of puppet especially with a huge install client base or
compute cluster etc. But come on guys. Is it too much task for simpler
more direct and meaningful error reporting?

Chris.

[Felix Frank]

Uhm, why is this in the "Sharing Puppet Functions" thread?

Anyway, this looks like a bug. Can you reproduce this? I'll gladly
report it if you'd like to save the hassle, but I'll need to reproduce.

Regards,
Felix

[Chris Johnson]

On 4/24/14, 11:22 AM, Felix Frank wrote:
> Uhm, why is this in the "Sharing Puppet Functions" thread?

Sorry, didn't know it belonged elsewhere. My bad.

>
> Anyway, this looks like a bug. Can you reproduce this? I'll gladly
> report it if you'd like to save the hassle, but I'll need to reproduce.

Very easily reproducible. I was reproducing it for four days. The
module I was trying to write initialized the /var/www/html Web content
directory with a index.html. Try mistyping something in the source path
that should normally go to the module's file directory (bloody
trifocals). BTW this is puppet and server 3.5.1 with current
dependencies on CentOS-6.5.

Chris

[José Luis Ledesma]

As John has said, there is the ppupetlabs/stdlib module the collects a lot of general function/ custom types.

Perhaps you can make a pull request with your development.

Stdlib is very spread along puppet users, so it could be very helpful.

Regards,

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/6c33bb67-d29d-4a11-9450-c545b469f655%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[William Leese]

On Thursday, April 24, 2014 10:36:13 PM UTC+9, jcbollinger wrote:

On Thursday, April 24, 2014 2:45:21 AM UTC-5, William Leese wrote:

The accepted standard for sharing code seems to be modules, which doesn't seem a nice fit if you simply want to share a single custom Puppet function.

Why not?

Let me be more clear; I mean sharing on the Forge. It seems overkill to write a module file and upload a tarball just to share 2 small files (function & rspec).

stdlib and some other modules provide a bundling of functions, when they're related (in function or acceptance by the community as standard library). But any 'bundle' without a clear designation is a poor fit for the forge because of lack of discoverability - I would have to add a tag for every keyword related to every function.

 

Having an easy way to look for Puppet functions would likely greatly help the development of such, providing a nice set of examples and avoiding duplicate work. 

I don't follow.  If you don't like modules for this purpose, then what would be the characteristics of your ideal solution?

(Again, this is just about sharing on the forge)

Perhaps a new type of resource on the Forge, with as big difference from the usual modules, that there's a fully searchable rubydoc and/or code. This way functions can easily be found that are developed by the community, that aren't coupled to a module (to install/manage X) per se.

 

Currently I simply create a module for each function and upload to the forge. Whats your approach? One monolithic module? Would that be easily findable on the Forge (tagging won't scale)? etc. etc.

Custom functions are valuable and widely used, but they are not a focal feature of Puppet.  It is uncommon for people to provide or seek a single custom function in isolation.  Instead, people usually are after a broader facility.  Typically that consists of classes and/or resource types (native or defined), possibly with associated custom functions.  In other words, a module.

I avoid custom functions when possible, knowing that the more custom stuff I use, the more troublesome upgrades can become :)

If you are writing custom functions so general that they are not associated with any particular system configuration regime, then it might be suitable to collect several into a library module.  PuppetLabs's 'stdlib' module follows this route (though it does provide a resource type or two along with the many functions).

I find that I've developed only about a handful of useful but completely unrelated functions in a year, despite using Puppet extensively for almost 2 years. 

Perhaps my problem isn't well represented in the community and as such there is little need. I wrote this post hoping to find more people frustrated by rarely finding the documentation or examples they need to write (more than basic) Puppet functions, having to resort to reading through Puppets code. 

[jcbollinger]

On Thursday, April 24, 2014 7:43:08 PM UTC-5, William Leese wrote:

On Thursday, April 24, 2014 10:36:13 PM UTC+9, jcbollinger wrote:

On Thursday, April 24, 2014 2:45:21 AM UTC-5, William Leese wrote:

The accepted standard for sharing code seems to be modules, which doesn't seem a nice fit if you simply want to share a single custom Puppet function.

Why not?

Let me be more clear; I mean sharing on the Forge. It seems overkill to write a module file and upload a tarball just to share 2 small files (function & rspec).

stdlib and some other modules provide a bundling of functions, when they're related (in function or acceptance by the community as standard library). But any 'bundle' without a clear designation is a poor fit for the forge because of lack of discoverability - I would have to add a tag for every keyword related to every function.

Indeed, a bundle of unrelated things does not really constitute a coherent module.  Moreover, essentially the same problems will attend such a collection whether it is organized as a Puppet module or in some other form.  They arise from the lack of commonality, not from the specific form of the collection.

But still, if you really do want to share individual custom functions then what's so bad about putting them into single-function modules?

 

 

Having an easy way to look for Puppet functions would likely greatly help the development of such, providing a nice set of examples and avoiding duplicate work. 

I don't follow.  If you don't like modules for this purpose, then what would be the characteristics of your ideal solution?

(Again, this is just about sharing on the forge)

Perhaps a new type of resource on the Forge, with as big difference from the usual modules, that there's a fully searchable rubydoc and/or code. This way functions can easily be found that are developed by the community, that aren't coupled to a module (to install/manage X) per se.

 

I really don't foresee any alternative to modules being developed for managing Puppet extensions.  Keep in mind that modules are not about the Forge; rather, the Forge is about modules.  Modules are THE way that Puppet manifests and plugins should be organized and managed.

On the other hand, I could imagine some new form of documenting module contents that could be automatically picked up and indexed by the Forge to facilitate discovery of modules based more directly on their contents.

 

Currently I simply create a module for each function and upload to the forge. Whats your approach? One monolithic module? Would that be easily findable on the Forge (tagging won't scale)? etc. etc.

Custom functions are valuable and widely used, but they are not a focal feature of Puppet.  It is uncommon for people to provide or seek a single custom function in isolation.  Instead, people usually are after a broader facility.  Typically that consists of classes and/or resource types (native or defined), possibly with associated custom functions.  In other words, a module.

I avoid custom functions when possible, knowing that the more custom stuff I use, the more troublesome upgrades can become :)

If you are writing custom functions so general that they are not associated with any particular system configuration regime, then it might be suitable to collect several into a library module.  PuppetLabs's 'stdlib' module follows this route (though it does provide a resource type or two along with the many functions).

I find that I've developed only about a handful of useful but completely unrelated functions in a year, despite using Puppet extensively for almost 2 years. 

Perhaps my problem isn't well represented in the community and as such there is little need. I wrote this post hoping to find more people frustrated by rarely finding the documentation or examples they need to write (more than basic) Puppet functions, having to resort to reading through Puppets code.

Yes, one of Puppet's weaknesses is the (limited amount of) documentation of its extension APIs.   Believe it or not, though, the docs are better than they used to be!  With respect to learning to write custom functions, I suspect that most people who want examples just pick up stdlib.  It is well known, is maintained by PL itself, and contains a great number and variety of functions.

Or, in fact, many modules on the Forge have a corresponding repository on GitHub.  That's often where I go when I want to look into the implementation of module components.


John

[Felix Frank]

So, I can sort of reproduce. I created a file at the URL
puppet:///modules/demo/var/www/index.html

Initially, I tried what you suggested and just mistyped my URL as
puppet:///modules/demo/ar/www/index.html

The error is humane enough:

Error: /Stage[main]/Main/File[/tmp/index.html]: Could not evaluate:
Could not retrieve information from environment production source(s)
puppet:///modules/demo/ar/www/index.html

So, from your earlier post I suspected that you hadn't just made *any*
typo, but had in fact skipped one of the slashes in puppet:///.

puppet://modules/demo/ar/www/index.html

Error: /Stage[main]/Main/File[/tmp/index.html]: Could not evaluate:

Could not retrieve file metadata for

puppet://modules/demo/ar/www/index.html: getaddrinfo: Name or service
not known

Lo and behold, and it even makes sense - after all, the domain in the
URL is now "modules", which doesn't resolve. Unfortunately for you, it
does resolve on *your* machine. To complete my test, I added a hosts
file entry for "modules" with a random working IP from my LAN and sure
enough:

Error: /Stage[main]/Main/File[/tmp/index.html]: Could not evaluate:

Could not retrieve file metadata for

puppet://modules/demo/ar/www/index.html: Connection refused - connect(2)

Btw, setting modules to be an alias for my master host:

Error: /Stage[main]/Main/File[/tmp/index.html]: Could not evaluate:

Could not retrieve file metadata for

puppet://modules/demo/ar/www/index.html: Server hostname 'modules' did
not match server certificate; expected one of geras.localdomain,
DNS:geras.localdomain, DNS:puppet, DNS:puppet.localdomain

So this would have likely been more helpful to you, but alas.

So, there is really no way for Puppet to tell that a) the URL is typoed
(it's valid and the domain name even resolves) and so b) assuming that
"modules" is your chosen fileserver, it has no choice but to notify you
of the fact that the Puppet port is not open.

Hope that's satisfactory, regards,

Felix

On 04/24/2014 05:40 PM, Chris Johnson wrote:
> Very easily reproducible. I was reproducing it for four days. The
> module I was trying to write initialized the /var/www/html Web content
> directory with a index.html. Try mistyping something in the source path
> that should normally go to the module's file directory (bloody
> trifocals). BTW this is puppet and server 3.5.1 with current
> dependencies on CentOS-6.5.