here is new resource type sysprop v1.0

[Philip Brown]

Here is a tool that lets Solaris sysadmins ensure that a particular SMF / svc property is set in a particular fashion.

In particular, I'm going to be using it to make sure that /network/smtp:sendmail  config/local_only = true

I've tried to code it so that it can handle all sorts of properties though.

I looked at the "forge" thing, and the "puppet-module" tool.

Sad to say, I did not like it, for this purpose. It seems too complex compared to the alternatives.

For the record, I deploy puppet by downloading the tarfiles, and running ruby install.rb

So.. I'm going to publicly release my sysprop type here for now. 

Future versions may probably be found on my website, http://www.bolthole.com, but I cant give a specific url for now, because I'm not sure whether I'm going to make a puppet/ subsection or not 

Depends on whether or not I write more providers.

Since I'm not sure how  google groups, or the actual mailing list, handles attachments, I'm going old school and using shar for this :)

If you have "unshar", just run it on this post. If you dont, then hand-trim out the top part and feed it directly to 'sh'.

If you like, (and you're very trusting :) you should be able to extract directly in the "puppet" dir, and the two files will go to where they belong.

# This is a shell archive.  Save it in a file, remove anything before

# this line, and then unpack it by entering "sh file".  Note, it may

# create directories; files and directories will be owned by you and

# have default permissions.

#

# This archive contains:

#

#       provider/sysprop/solaris.rb

#       type/sysprop.rb

#

echo x - provider/sysprop/solaris.rb

mkdir -p provider/sysprop 2>/dev/null

sed 's/^X//' >provider/sysprop/solaris.rb << 'END-of-provider/sysprop/solaris.rb'

XPuppet::Type.type(:sysprop).provide(:solaris) do

X  desc "Solaris provider for svcprop type"

X  defaultfor :operatingsystem => :solaris

X

X  commands :svccfg => "/usr/sbin/svccfg"

X  commands :svcprop => "/usr/bin/svcprop"

X  commands :svcadm => "/usr/sbin/svcadm"

X

X  def create

X    svccfg("-s",resource[:fmri],"setprop",resource[:property],"=",resource[:value])

X    refresh="/usr/sbin/svcadm refresh #{@resource[:fmri]}"

X    execute(refresh)

X  end

X  def destroy

X    raise ArgumentError, "sysprop cant really destroy a property. Define a value for it instead"

X  end

X

X  def exists?

X    if ("#{@resource[:property]}" == "")

X      raise ArgumentError, "sysprop requires 'property' to be defined"

X    end

X    if ("#{@resource[:value]}" == "")

X      raise ArgumentError, "sysprop requires 'value' to be defined"

X    end

X

X    output = svcprop("-p", @resource[:property], @resource[:fmri]).strip

X    #Puppet.debug "svcprop return value is '#{output}'"

X

X    if output == "#{@resource[:value]}"

X      return true

X    end

X    

X    return false

X  end

X

Xend

X

X

END-of-provider/sysprop/solaris.rb

echo x - type/sysprop.rb

mkdir type 2>/dev/null

sed 's/^X//' >type/sysprop.rb << 'END-of-type/sysprop.rb'

Xmodule Puppet

X  newtype(:sysprop) do

X    @doc = "Handle `system properties`. Inspired by solaris svcprop, and

X      primarily implemented for that purpose, although could theoretically have

X      providers for other systems as well

X

X

X      Sample usage:

X        sysprop {

X         "solaris-localvar":

X         ensure=>present,

X         fmri=>"network/smtp:sendmail",

X         property=>"config/local_only",

X         value=>true,

X       }

X

X      Note:

X       - You must have "ensure=>present" for it to work.

X"

X

X    ensurable do

X      defaultvalues

X      def change_to_s(oldstate,newstate)

X        if ( oldstate == :absent && newstate == :present )

X          return "set #{@resource[:property]} = #{@resource[:value]}"

X        end

X      end

X    end

X

X    newparam(:name) do

X      desc "Label for this resource"

X      isnamevar

X    end

X    newparam(:fmri) do

X      # Do NOT make this 'namevar'. This way, it allows the

X      # setting of multiple properties for a single fmri

X      desc "FMRI that the property is associated with"

X    end

X    newparam(:property) do

X      desc "Specific name of the property to be set or checked for the FMRI"

X    end

X    newparam(:value) do

X      desc "Value to set/check for the named property"

X    end

X    

X  end

Xend

END-of-type/sysprop.rb

exit

[Ken Barber]

> Here is a tool that lets Solaris sysadmins ensure that a particular SMF /
> svc property is set in a particular fashion.
> In particular, I'm going to be using it to make sure that
> /network/smtp:sendmail  config/local_only = true
> I've tried to code it so that it can handle all sorts of properties though.

Awesome.

> I looked at the "forge" thing, and the "puppet-module" tool.
> Sad to say, I did not like it, for this purpose. It seems too complex
> compared to the alternatives.

Do you really think its hard? I was under the impression it was
relatively easy these days.

From a user perspective its really just about doing 'puppet module
install foobar-modulename' ...

> For the record, I deploy puppet by downloading the tarfiles, and running
> ruby install.rb
>
> So.. I'm going to publicly release my sysprop type here for now.
> Future versions may probably be found on my website,
> http://www.bolthole.com, but I cant give a specific url for now, because I'm
> not sure whether I'm going to make a puppet/ subsection or not
> Depends on whether or not I write more providers.
> Since I'm not sure how  google groups, or the actual mailing list, handles
> attachments, I'm going old school and using shar for this :)
> If you have "unshar", just run it on this post. If you dont, then hand-trim
> out the top part and feed it directly to 'sh'.

I know there is no school like the old school - but this sounds much
much harder then using the Puppet Module tool - IMHO :-).

> If you like, (and you're very trusting :) you should be able to extract
> directly in the "puppet" dir, and the two files will go to where they
> belong.

So okay - the way I read this was that the tool is to be unpacked
inside the Puppet core source files? Am I mistaken here? Ordinarily
'modules' are kept away from the core files - as plugins shouldn't be
mixed with core.

ken.

[Philip Brown]

On Sunday, May 6, 2012 8:19:21 AM UTC-7, Ken Barber wrote:

> I looked at the "forge" thing, and the "puppet-module" tool.
> Sad to say, I did not like it, for this purpose. It seems too complex
> compared to the alternatives.

Do you really think its hard? I was under the impression it was
relatively easy these days.

huh. I dunno, lets just say that it did not seem that way at

https://github.com/puppetlabs/puppet-module-tool/blob/master/README.markdown 

If I have to go trawl through over 1,000(?) words, and scroll down multiple pages, to find out how to do something that is supposed to be "easy"... (and still havent even gotten to the details), I start looking elsewhere.

I suspect I'm not alone in this :)

So okay - the way I read this was that the tool is to be unpacked 
inside the Puppet core source files? Am I mistaken here? Ordinarily 
'modules' are kept away from the core files - as plugins shouldn't be 
mixed with core. 

 

So okay - the way I read this was that the tool is to be unpacked 

inside the Puppet core source files? Am I mistaken here? Ordinarily 

'modules' are kept away from the core files - as plugins shouldn't be 

mixed with core. 

Huh.  well since the details of this werent mentioned in 

http://docs.puppetlabs.com/guides/complete_resource_example.html

it kinda fell of my radar. Would be nice if it had a very brief, focused mention of this here.

Comment on that though:

It seems like, from http://docs.puppetlabs.com/guides/plugins_in_modules.html

that it would be expected to install under

/var/lib/puppet/lib/modulename/lib/puppet/

Could we be less redundantly redundant, please? :-}

Yikes....  :(

Chopping off the trailing "lib/puppet" stuff, seems to be more reasonable. 

Seems like the primary reason yoy do that is for facter extensions. But facter stuff, belongs with facter, not under a "puppet" dir.

It has its own directory under ruby site_ruby, after all. And it's own "Separate" installation tarfile, etc, etc.

Not to mention that, while I consider classes, etc. to be part of the "manifest" stuff... I consider stuff that is actually *ruby code*, rather than puppet code, to belong with the ruby code.

Mixing the two feels extremely wrong to me, from my CompSci background.

/var/blah  and /etc/blah are for the manifests & classes: $RUBYDIR/site_ruby is for ruby code, imo.

How about a ruby_gems way to distribute that stuff instead, that collects it under the ruby/puppet tree somewhere?

[Ken Barber]

>> > I looked at the "forge" thing, and the "puppet-module" tool.
>> > Sad to say, I did not like it, for this purpose. It seems too complex
>> > compared to the alternatives.
>>
>> Do you really think its hard? I was under the impression it was
>> relatively easy these days.
>
>
> huh. I dunno, lets just say that it did not seem that way at
> https://github.com/puppetlabs/puppet-module-tool/blob/master/README.markdown
>
> If I have to go trawl through over 1,000(?) words, and scroll down multiple
> pages, to find out how to do something that is supposed to be "easy"... (and
> still havent even gotten to the details), I start looking elsewhere.
> I suspect I'm not alone in this :)

So the principle is:

* Create a Modulefile according to the defined format in that doc
* Run 'puppet module generate' (or puppet-module generate if your
using the old tool)
* Create a forge account
* Create a project
* Create a release
* Upload the previously created tarball from pkg/ into your release.

(Take note that the PMT module is available built-in to Puppet with
modern releases of Puppet - so you won't need that separate tool any
more.)

Once this happens you should be able to do:

* puppet module search <module_name>
* puppet module install <module_name>

If all is working well ...

>>> So okay - the way I read this was that the tool is to be unpacked
>>>
>>> inside the Puppet core source files? Am I mistaken here? Ordinarily
>>>
>>> 'modules' are kept away from the core files - as plugins shouldn't be
>>>
>>> mixed with core.
>>>
>>>
>
>
> Huh.  well since the details of this werent mentioned in
> http://docs.puppetlabs.com/guides/complete_resource_example.html
> it kinda fell of my radar. Would be nice if it had a very brief, focused
> mention of this here.
>
> Comment on that though:
> It seems like,
> from http://docs.puppetlabs.com/guides/plugins_in_modules.html
> that it would be expected to install under
> /var/lib/puppet/lib/modulename/lib/puppet/
>
> Could we be less redundantly redundant, please? :-}
> Yikes....  :(
>
> Chopping off the trailing "lib/puppet" stuff, seems to be more reasonable.
> Seems like the primary reason yoy do that is for facter extensions. But
> facter stuff, belongs with facter, not under a "puppet" dir.
> It has its own directory under ruby site_ruby, after all. And it's own
> "Separate" installation tarfile, etc, etc.

Actually this isn't correct ... on the puppet master modules should
live in the modulepath directory, which for the OSS version is
usually:

/etc/puppet/modules/<module_name>

Then when clients run they get 'plugin-sync'd to the 'libdir' which is:

/var/lib/puppet/lib/<puppet|facter>/...

Uploading your module to the forge means the download and placement of
code into the module path should (with a good setup) go into the right
place, so a lot of this pain isn't felt by the user.

> Not to mention that, while I consider classes, etc. to be part of the
> "manifest" stuff... I consider stuff that is actually *ruby code*, rather
> than puppet code, to belong with the ruby code.
> Mixing the two feels extremely wrong to me, from my CompSci background.
> /var/blah  and /etc/blah are for the manifests & classes: $RUBYDIR/site_ruby
> is for ruby code, imo.
> How about a ruby_gems way to distribute that stuff instead, that collects it
> under the ruby/puppet tree somewhere?

Lets get your module working the preferred way first and then you can
evaluate its pluses/minuses under that context :-).

ken.

[Nan Liu]

First off, thanks for taking the time to write and share your smf
property provider. I've list of some comments/suggestions below.

Maybe confine to solaris as well?

> X  commands :svccfg => "/usr/sbin/svccfg"
> X  commands :svcprop => "/usr/bin/svcprop"
> X  commands :svcadm => "/usr/sbin/svcadm"

You don't have to supply the fully qualified path to the command.

> X  def create
> X
>  svccfg("-s",resource[:fmri],"setprop",resource[:property],"=",resource[:value])
> X    refresh="/usr/sbin/svcadm refresh #{@resource[:fmri]}"
> X    execute(refresh)

Any reason not using?

svcadm('refresh', @resource[:frmri])

You can add defaultto :present in ensurable to avoid requiring users
specify ensure => present.

If you are looking for an example an a puppet type/provider installed
via a module, you can check out:
https://github.com/puppetlabs/puppetlabs-firewall

Thanks,

Nan

[Philip Brown]

On Sunday, May 6, 2012 2:09:20 PM UTC-7, Ken Barber wrote:

>> > I looked at the "forge" thing, and the "puppet-module" tool.
>> > Sad to say, I did not like it, for this purpose. It seems too complex
>> > compared to the alternatives.
>>
>> Do you really think its hard? I was under the impression it was
>> relatively easy these days.
>
>
> huh. I dunno, lets just say that it did not seem that way at
> https://github.com/puppetlabs/puppet-module-tool/blob/master/README.markdown
>
> If I have to go trawl through over 1,000(?) words, and scroll down multiple
> pages, to find out how to do something that is supposed to be "easy"... (and
> still havent even gotten to the details), I start looking elsewhere.
> I suspect I'm not alone in this :)

So the principle is:

* Create a Modulefile according to the defined format in that doc
* Run 'puppet module generate' (or puppet-module generate if your
using the old tool)

I am using a puppet 2.7.9, and ruby 1.8.7

So, I cant use these moduley things, unless I additionally install "puppet-module".

I cant install an officially "released" version of puppet-module , unless I install "gems", it seems. (which, oddly, is not part of core ruby)

I cant even install a source version, unless I install git... and even then, there's no nice "install.rb" file like puppet and facter have... I have to apparently install yet ANOTHER tool, rake?

:(

Lemme know when your "easy module installation tool" is actually easy to install itself, please.

Having a one-stop-shopping install.rb file would be one way to accomplish this. It's rather backwards, when your "easy" puppet configuration tool, is more difficult to install than puppet itself.

No, sadly, I cant upgrade our version of puppet at this time.

[Philip Brown]

On Monday, May 7, 2012 8:06:22 AM UTC-7, Nan Liu wrote:

First off, thanks for taking the time to write and share your smf
property provider. I've list of some comments/suggestions below.

...

>  svccfg("-s",resource[:fmri],"setprop",resource[:property],"=",resource[:value])
> X    refresh="/usr/sbin/svcadm refresh #{@resource[:fmri]}"
> X    execute(refresh)

Any reason not using?

svcadm('refresh', @resource[:frmri])

oops.

 

You can add defaultto :present in ensurable to avoid requiring users
specify ensure => present.

Thanks for the tips, Nan

 

[James Turnbull]

Philip Brown wrote:
> So, I cant use these moduley things, unless I additionally install
> "puppet-module".
> I cant install an officially "released" version of puppet-module ,
> unless I install "gems", it seems. (which, oddly, is not part of core ruby)
> I cant even install a source version, unless I install git... and even
> then, there's no nice "install.rb" file like puppet and facter have... I
> have to apparently install yet ANOTHER tool, rake?
>
> :(
>
> Lemme know when your "easy module installation tool" is actually easy to
> install itself, please.
> Having a one-stop-shopping install.rb file would be one way to
> accomplish this. It's rather backwards, when your "easy" puppet
> configuration tool, is more difficult to install than puppet itself.
>

Actually it's not - packages work and are the recommended way anyone
should manage things like this:

$ yum/apt install ruby rubygems
$ gem install puppet-module

Installing from source IMHO just doesn't scale.

Our even better:

package { [ "ruby", "rubygems" ]:
ensure => latest,
}

package { "puppet-module":
ensure => latest,
provider => gem,
require => Package["rubygems"],
}

Easy.

James

--
James Turnbull
Puppet Labs
1-503-734-8571
To schedule a meeting with me: http://tungle.me/jamtur01

[Philip Brown]

On Tuesday, May 8, 2012 3:34:58 PM UTC-7, James Turnbull wrote:

Philip Brown wrote:
>
> Lemme know when your "easy module installation tool" is actually easy to
> install itself, please.
> Having a one-stop-shopping install.rb file would be one way to
> accomplish this. It's rather backwards, when your "easy" puppet
> configuration tool, is more difficult to install than puppet itself.
>

Actually it's not - packages work and are the recommended way anyone
should manage things like this:

$ yum/apt install ruby rubygems

Too bad that all the world isnt linux.

I dont have yum. I dont have apt.

Puppet is supposed to be cross platform. Please keep the installation easy for more than just linux.

 

[James Turnbull]

Philip Brown wrote:

> Too bad that all the world isnt linux.
> I dont have yum. I dont have apt.
> Puppet is supposed to be cross platform. Please keep the installation
> easy for more than just linux.
>

package { [ "ruby", "rubygems" ]:
ensure => latest,
}

package { "puppet-module":
ensure => latest,
provider => gem,
require => Package["rubygems"],
}

Still easy.

Regards

James


--
James Turnbull
Puppet Labs
1-503-734-8571

To schedule a meeting with me: http://tungle.me/jamtur01

[Philip Brown]

On Sunday, May 6, 2012 2:09:20 PM UTC-7, Ken Barber wrote:

>> Do you really think its hard? I was under the impression it was
>> relatively easy these days.
>
>
> huh. I dunno, lets just say that it did not seem that way at
> https://github.com/puppetlabs/puppet-module-tool/blob/master/README.markdown
>

So the principle is:

* Create a Modulefile according to the defined format in that doc
* Run 'puppet module generate' (or puppet-module generate if your
using the old tool)

...

Once again, I hit a "documentation is lacking" problem.

The above command generates a whole BUNCH of files:
./tests/init.pp
./README
./Modulefile
./spec/spec.opts
./spec/spec_helper.rb
./metadata.json
./manifests/init.pp

But the README is utterly useless in mentioning what the contents of each of those files is supposed to be.

Also... my earlier assumption seems to be correct. In principle, at least.
Just looking at the "best practices layout" autogenerated there... this seems to be primarily geared towards *classes*, and end user "modules".
Not back end "here's a whole new resource type" code.

And IMO, that's as it should be. Back end ruby language code should be held separate from regular "puppet language code" stuff.

But that doesn't leave me a clear path for how to have my svcprop resource shared out to others.

As a suggestion.. if you guys INSIST on overloading the term "module", to cover both user-level code, and back-end code, then I would recommend you add an additional, separate subcommand to puppet-module, that generates back-end templating, instead of the default user-side templating.
Dumping both types of templating as the default, would make things more confusing to the general-case user.
It's confusing enough (with all the extra junk like "metadata.json" and "./spec/spec.opts") without making it more so.

[Peter Meier]

> Also... my earlier assumption seems to be correct. In principle, at least.
> Just looking at the "best practices layout" autogenerated there... this
> seems to be primarily geared towards *classes*, and end user "modules".
> Not back end "here's a whole new resource type" code.

A module is more than just classes. These classes might need the
resource types that this module bring with. And therefore I don't see
a reason why plugins (you call it back-end code) shouldn't come with
modules as well.

Afair the whole idea of what a module is is pretty well described in
the documentation.

> And IMO, that's as it should be. Back end ruby language code should be held
> separate from regular "puppet language code" stuff.

It is - in the plugin aka. lib/ directory of a module.

> But that doesn't leave me a clear path for how to have my svcprop resource
> shared out to others.
>
> As a suggestion.. if you guys INSIST on overloading the term "module", to
> cover both user-level code, and back-end code, then I would recommend you
> add an additional, separate subcommand to puppet-module, that generates
> back-end templating, instead of the default user-side templating.

It is not that easy to say, that all puppet code is user-level code.
There might different levels of abstraction and you might end up with
puppet code that has the same functionality as what you are calling
back-end code.

~pete

[Ken Barber]

> Once again, I hit a "documentation is lacking" problem.
>
> The above command generates a whole BUNCH of files:
> ./tests/init.pp
> ./README
> ./Modulefile
> ./spec/spec.opts
> ./spec/spec_helper.rb
> ./metadata.json
> ./manifests/init.pp

Yeah - my bad, I'm sorry - try 'puppet module build' after creating a
Module file. Puppet module generate wasn't what you wanted - although
you've seen now how it can be used to generate at least a basic
template.

> But the README is utterly useless in mentioning what the contents of each of
> those files is supposed to be.
>
> Also... my earlier assumption seems to be correct. In principle, at least.
> Just looking at the "best practices layout" autogenerated there... this
> seems to be primarily geared towards *classes*, and end user "modules".
> Not back end "here's a whole new resource type" code.

I think we aim there for the common case. See my comments at the end
of the email.

> And IMO, that's as it should be. Back end ruby language code should be held
> separate from regular "puppet language code" stuff.

I'm still not convinced, but I'm open to being convinced :-).

Both have 'backend' in this sense surely - however the water is a lot
more murky and not so black & white as you might think. Puppet DSL
(classes & defines resources) compile on a server, resource types at
least run on the server - whereas resource providers are executed on
the client. Facts are client side, functions are server side
(usually). But the goal is to have a bundle so when its loaded onto
the master, client & server side bits are in one place - and the
'client' side bits are distributed correctly.

A complete module, might contain some resource types + some classes to
help install it. The puppetlabs-opennebula module at least contains
components of all kinds: facts, resources, and DSL. These are all
needed to support 'opennebula' and while they could be separated - the
entire suite is kind of needed for things to function. I don't think
functionally it would make sense it breaks this up at all ...

I think you can't make clear divisions about the language as to how it
gets sliced up either - things get sliced up in the real world on
functionality not language type ... take functions for example. These
are ruby extensions, but more often then not I'm combining them with a
module to provide functionality specific to that module. A fact about
the version of opennebula for example - well it belongs next to the
Opennebula setup class - because its the consumer surely?

> But that doesn't leave me a clear path for how to have my svcprop resource
> shared out to others.
>
> As a suggestion.. if you guys INSIST on overloading the term "module", to
> cover both user-level code, and back-end code, then I would recommend you
> add an additional, separate subcommand to puppet-module, that generates
> back-end templating, instead of the default user-side templating.
> Dumping both types of templating as the default, would make things more
> confusing to the general-case user.
> It's confusing enough (with all the extra junk like "metadata.json" and
> "./spec/spec.opts") without making it more so.

This is a fair enough feature request - I would presume there are
almost 'template styles' that one would want. ie. Generate a templated
resource, generate a class, generate a fact or function etc. So that
way, you could almost have people defining their own set of templates
as well. I often find that would help a customer, when they have
finally decided on a preferred template - giving them the capability
to keep their own library would be awesome. What do you think?

ken.

[Philip Brown]

On Wednesday, May 9, 2012 2:23:46 PM UTC-7, Ken Barber wrote:

>
> And IMO, that's as it should be. Back end ruby language code should be held
> separate from regular "puppet language code" stuff.

I'm still not convinced, but I'm open to being convinced :-).

Here's the important factor, which you guys @puppetlabs simply cannot see, because you've been steeped in it so long:

"puppet language", and ruby, are *two* *separate* *languages*.
Similar, yet different. Each with slightly different quirks. And in addition to that, there's (plain ruby), and then there's (ruby + puppet back-end code).
Which is a smaller scale equivalent of (C language, plus insert-toolkit-lib-here)

Try to picture a sysadmin. a reasonably competant sysadmin. But one who has zero prior experience with ruby, or puppet.

Now try to imagine this person attempting to fully understand puppet.

To *fully* undestand it, he has to try to learn 2.5 brand new languages.

Which is made even more difficult by the "minor detail", that
NO-ONE TELLS HIM THEY ARE DIFFERENT.

This is enough to make even a highly competant sysadmin run away screaming.

In fact, I myself did, some years ago. But I've had enough motivation to come back and give it another go lately.

Point of reference:
I've been in the industry for 20 years.
I have a computer science degree.
I know approximately 10 diferent computer languages, and 3 real world ones.
I've written device drivers, and 5,000 line shell scripts.

if it was difficult for ME to understand and separate, contextually... how difficult do you think it will be for the average sysadmin who has zero ruby experience, etc?


Because of this, I'm saying it is really really important to have clearly defined borders between "frontend" puppet language
 (the language of manifests)
and backend (ruby, and related puppet code)
Simply for the sanity of any poor admin who attempts to read this stuff, and doesnt realize he's stepped over into an entirely different language country, when he changed files.
Because after all, "they're all just a 'puppet module', right? so they're all the same language".

[Ken Barber]

>> > And IMO, that's as it should be. Back end ruby language code should be
>> > held
>> > separate from regular "puppet language code" stuff.
>>
>> I'm still not convinced, but I'm open to being convinced :-).
>>
>
> Here's the important factor, which you guys @puppetlabs simply cannot see,
> because you've been steeped in it so long:
>
> "puppet language", and ruby, are *two* *separate* *languages*.
> Similar, yet different. Each with slightly different quirks. And in addition
> to that, there's (plain ruby), and then there's (ruby + puppet back-end
> code).
> Which is a smaller scale equivalent of (C language, plus
> insert-toolkit-lib-here)
>
> Try to picture a sysadmin. a reasonably competant sysadmin. But one who has
> zero prior experience with ruby, or puppet.
>
> Now try to imagine this person attempting to fully understand puppet.
>
> To *fully* undestand it, he has to try to learn 2.5 brand new languages.
>
> Which is made even more difficult by the "minor detail", that
> NO-ONE TELLS HIM THEY ARE DIFFERENT.

I would have thought that file extensions were enough to indicate
language change. Puppet manifests end in 'pp' and ruby files end in
'rb'.

Also - be mindful - that puppet code lives in 'manifests'. And ruby
code lives in 'lib'. We could always rename 'lib' to 'ruby', but we
didn't want to be specific.

> This is enough to make even a highly competant sysadmin run away screaming.

I don't think that pure ruby resources are necessarily aimed at
sysadmins - although one day I would love to simplify them to such an
extent. They are today, without a doubt - the more difficult extension
within Puppet and do require some reasonable Ruby knowledge to write.
In fact we do dedicate for this reason most of our development
training course to this.

The alternative however is to simply use the lighter 'defined
resource' approach which is a pure Puppet DSL solution to writing
resources, which often gets a reasonable mileage - albeit you lack
some nicer features.

> In fact, I myself did, some years ago. But I've had enough motivation to
> come back and give it another go lately.

I'm glad you have given it a second chance.

> Point of reference:
> I've been in the industry for 20 years.
> I have a computer science degree.
> I know approximately 10 diferent computer languages, and 3 real world ones.
> I've written device drivers, and 5,000 line shell scripts.
>
> if it was difficult for ME to understand and separate, contextually... how
> difficult do you think it will be for the average sysadmin who has zero ruby
> experience, etc?

Again - I don't think we hide the fact that resources aren't for the
'average sysadmin' to write. Its a terrible shame - but there we are.
I wouldn't mind at some point a relook at the API for writing
resources, but they are still in effect Ruby components and follow
ruby rules.

I have pondered the value in fact of extending the defined resource
concept in Puppet DSL to extend its capabilities so it gets closer to
pure ruby resources.

But yes - as you have stated it sounds more like we need to extend the
documentation for writing resources a little more at least so people
have good examples to follow along to.

> Because of this, I'm saying it is really really important to have clearly
> defined borders between "frontend" puppet language
>  (the language of manifests)
> and backend (ruby, and related puppet code)
> Simply for the sanity of any poor admin who attempts to read this stuff, and
> doesnt realize he's stepped over into an entirely different language
> country, when he changed files.
> Because after all, "they're all just a 'puppet module', right? so they're
> all the same language".

Puppet code ends in .pp, ruby in .rb - manifest code belongs in
'manifests' and Ruby code belongs in 'lib'. Is there need for further
clarification?

Btw - templates belong in 'templates' and files in 'files'. Although
these are more aptly named :-).

ken.

[Philip Brown]

On Wednesday, May 9, 2012 2:23:46 PM UTC-7, Ken Barber wrote:

>
> As a suggestion.. if you guys INSIST on overloading the term "module", to
> cover both user-level code, and back-end code, then I would recommend you
> add an additional, separate subcommand to puppet-module, that generates
> back-end templating, instead of the default user-side templating.
> Dumping both types of templating as the default, would make things more
> confusing to the general-case user.
> It's confusing enough (with all the extra junk like "metadata.json" and
> "./spec/spec.opts") without making it more so.

This is a fair enough feature request - I would presume there are
almost 'template styles' that one would want. ie. Generate a templated
resource, generate a class, generate a fact or function etc. So that
way, you could almost have people defining their own set of templates
as well. I often find that would help a customer, when they have
finally decided on a preferred template - giving them the capability
to keep their own library would be awesome. What do you think?

Yup, something like that would be great. With the exception that I think you should limit it to *not* be user extensible. IMO, best to keep it to "officially blessed" organization only.

If they want to do their own templating, they can already. That's what the "jump straight to puppet module build" thing that you suggested to me, is for, combined with their own git repository or something.

But meanwhile, I have no template, so Im' still wondering where I put my type and resource provider files in this thing :-}
"puppet module build" just seemed to create a "pkg" directory, and a tarfile.
Without some output like the "generate" stuff, I still have been given no clue where or how it determines the contents for the tarfile, so I have no idea how to even get my files into it, let alone what the right place for them are. :(


 

[Ken Barber]

> But meanwhile, I have no template, so Im' still wondering where I put my
> type and resource provider files in this thing :-}
> "puppet module build" just seemed to create a "pkg" directory, and a
> tarfile.
> Without some output like the "generate" stuff, I still have been given no
> clue where or how it determines the contents for the tarfile, so I have no
> idea how to even get my files into it, let alone what the right place for
> them are. :(

So the top level location of stuff is defined here:

http://docs.puppetlabs.com/puppet/2.7/reference/modules_fundamentals.html

And then zooming into the 'lib' stuff:

http://docs.puppetlabs.com/guides/plugins_in_modules.html

You want your type here:

<module_name>/type/<type_name>.rb

And provider here:

<module_name>/provider/<type_name>/<provider_name>.rb

And then make sure your Modulefile has all the correct metadata. It
should be located here:

<module_name>/Modulefile

Find an example here:
https://github.com/puppetlabs/puppetlabs-opennebula/blob/master/Modulefile

And that's the minimum you'll need as far as I'm aware - just those 3
files. Everything else is optional.

Now, running:

puppet module build

Will generate the tarball in:

<module_name>/pkg/<module_name>_<revision>.tar.gz

And thats the bit you want to upload to the forge. Once its uploaded
you should be able to use the module tool to install it.

ken.

[Philip Brown]

Sooo may comments I have on this stuff! :-/  I'll try to keep them focused to just what Ken wrote.
Mostly :-)

On Wednesday, May 9, 2012 3:08:04 PM UTC-7, Ken Barber wrote:

So the top level location of stuff is defined here:

http://docs.puppetlabs.com/puppet/2.7/reference/modules_fundamentals.html

When I first came across puppet, couple years ago, I came across the term "module", and at the time, I thought it was just a higher level grouping of "class".
resource << class << module
And in fact, some some ways, that doc you just referenced, agrees with that.
"On disk, a module is simply a directory with a specific, predictable structure:"

Given that the path I took to reach the "modules" bit, had just been talking about pure user-side, non-puppet-back-end stuff, I still thought I was in pure puppet language land.

There is only a brief mention in the top half of the page, about the lib directory. it says,

"Contains plugins, like custom facts and custom resource types"

Umm. okay. (files that in back of brain, doesnt sound very crucial or different)

And with that, an assuption has just been made, "the lib directory is just like everything else (ie: same language), it just has a slightly different role to play"

Other parts of the doc support this assumption.

"To write a module, simply write classes and defined types and place them in properly named manifest files as described above."

Well, great! You just said it was "simple"! So it's easy!  CERTAINLY then, I would have to do anything COMPLICATED, like learn *two different languages* or anything. Clearly, it's "simple"
:-P :-P :-P~~~

Nothing on that page gives any direct indication, "beware! lib directory is totally different from all other directories"

There's also the problem of documentation fragmentation.
Just as the problem of code fragmentation, where any time you start maintaining parallel functionality in two separate source code files, they start to diverge... when you start maintaining parallel documentation on a subject it starts to diverge, and one inevitably starts falling beind, etc.

You have (at least!) two separate places for documenting what modules are and how to use/make them.
The stuff on forge.puppetlabs.com, and this stuff you referenced. There needs to be a single canonical place.

* * * Worst of all, you have different file layouts for the "same" thing.

The doc you just have the url for, above, says
( custom types go in   (MODULEDIR)/lib/(?somewhere?)  -- hey another bug, it doesnt say specifically where under lib it goes )

Whereas the layout you gave me previously  (hmm, and you actually repeat later on in the message I'm replying to) says that custom types should go in
(MODULEDIR)/type

Inconsistent!  Inconsistency== bad and confusing!!!  :(

(PS: yes I agree with your other thought, that renaming "lib" to "rubyext" or some such, could potentially be helpful to aid in disambiguating this stuff.

[Ken Barber]

> The doc you just have the url for, above, says
> ( custom types go in   (MODULEDIR)/lib/(?somewhere?)  -- hey another bug, it
> doesnt say specifically where under lib it goes )

On the page I sent you:

http://docs.puppetlabs.com/puppet/2.7/reference/modules_fundamentals.html

Under the lib directory it states:

Contains plugins, like custom facts and custom resource types. See
“Using Plugins” for more details.

Which redirects you to a hyperlink explaining the layout:

http://docs.puppetlabs.com/guides/plugins_in_modules.html

I think your tone is a little trollish Philip - can we be more civil
please? I'm just trying to help. If you want to raise concrete bugs on
docs or code, I'm happy to show you were these can be raised - or if
you want healthy debates about making things less ambiguous, I'm happy
to participate - but you have to be more polite with your tone.

ken.

[Philip Brown]

On Thursday, May 10, 2012 10:10:56 AM UTC-7, Ken Barber wrote:

I think your tone is a little trollish Philip - can we be more civil
please? I'm just trying to help.

 
Okay, sorry. One man's humor is another man's trolling, I suppose. So here's plain and boring:

 

> ( custom types go in   (MODULEDIR)/lib/(?somewhere?)  -- hey another bug, it
> doesnt say specifically where under lib it goes )

On the page I sent you:

http://docs.puppetlabs.com/puppet/2.7/reference/modules_fundamentals.html

Under the lib directory it states:

Contains plugins, like custom facts and custom resource types. See
“Using Plugins” for more details.

Which redirects you to a hyperlink explaining the layout:

http://docs.puppetlabs.com/guides/plugins_in_modules.html

I understand that on the "deeper" linked page, it may clear things up. But most people dont read manuals from cover to cover. Nor do they normally do a "depth first parsing" of documentation trees. They tend to first read the bits that seem to be most relevant to them.
 
The top level needs to more noticably spell out to the reader, "hey pay attention, this bit(lib dir) is very different, you should go read the detail page to understand it more."

 

If you want to raise concrete bugs on
docs or code, I'm happy to show you were these can be raised -

There is also the "concrete bug" of the inconsistency of location I mentioned, for custom types.

[Ken Barber]

>> On the page I sent you:
>>
>> http://docs.puppetlabs.com/puppet/2.7/reference/modules_fundamentals.html
>>
>> Under the lib directory it states:
>>
>> Contains plugins, like custom facts and custom resource types. See
>> “Using Plugins” for more details.
>>
>> Which redirects you to a hyperlink explaining the layout:
>>
>> http://docs.puppetlabs.com/guides/plugins_in_modules.html
>
>
> I understand that on the "deeper" linked page, it may clear things up. But
> most people dont read manuals from cover to cover. Nor do they normally do a
> "depth first parsing" of documentation trees. They tend to first read the
> bits that seem to be most relevant to them.
>
> The top level needs to more noticably spell out to the reader, "hey pay
> attention, this bit(lib dir) is very different, you should go read the
> detail page to understand it more."

Sounds fair enough, if you think you have a clear idea of how you
would like the prose to be changed on that particular docs page, put a
proposal together and submit it as a ticket in Redmine:

http://projects.puppetlabs.com/projects/puppet-docs

Where it can be discussed.

>> If you want to raise concrete bugs on
>> docs or code, I'm happy to show you were these can be raised -
>
>
> There is also the "concrete bug" of the inconsistency of location I
> mentioned, for custom types.

What bug do you mean? The path to types?

The location to types is documented here:

http://docs.puppetlabs.com/guides/plugins_in_modules.html

"custom types should go in lib/puppet/type/"

What I wrote in email was wrong yes - what I meant to say was:

<snip>

You want your type here:

<module_name>/lib/puppet/type/<type_name>.rb

And provider here:

<module_name>/lib/puppet/provider/<type_name>/<provider_name>.rb
</snip>

I was writing that from scratch - not copying from docs - I just
truncated the path - sorry about that.

ken.

[Philip Brown]

On Thursday, May 10, 2012 11:20:23 AM UTC-7, Ken Barber wrote:

>
> The top level needs to more noticably spell out to the reader, "hey pay
> attention, this bit(lib dir) is very different, you should go read the
> detail page to understand it more."

Sounds fair enough, if you think you have a clear idea of how you
would like the prose to be changed on that particular docs page, put a
proposal together and submit it as a ticket in Redmine:

http://projects.puppetlabs.com/projects/puppet-docs

Okay, done. URL = http://projects.puppetlabs.com/issues/14412


Two related questions/comments:

1. How do I address the problem of forge vs docs site redundancy? Not sure that the above redmine area is the correct place

2. The trouble with the bug I filed, though.. is that personally, I think the documentation in that area could do with a *Major* rewrite.
I filed some stopgap measures, but a rewrite is called for.
It would not make sense to do that until the "forge vs docs site" issue is decently resolves, though.

Additionally, I do have some experience writing online documentation, and could potentially contribute.
The bug system is not adequate for doing that, though.

[Ken Barber]

> 1. How do I address the problem of forge vs docs site redundancy? Not sure
> that the above redmine area is the correct place

Its a good enough place to start. If the documentation overlaps or
says two different things its a fair place to cover it.

> 2. The trouble with the bug I filed, though.. is that personally, I think
> the documentation in that area could do with a *Major* rewrite.
> I filed some stopgap measures, but a rewrite is called for.
> It would not make sense to do that until the "forge vs docs site" issue is
> decently resolves, though.
>
> Additionally, I do have some experience writing online documentation, and
> could potentially contribute.
> The bug system is not adequate for doing that, though.

On that URL I gave you:

http://projects.puppetlabs.com/projects/puppet-docs

There is a link on how to contribute:

http://docs.puppetlabs.com/contribute.html

ken.

[Philip Brown]

On Thursday, May 10, 2012 11:56:15 AM UTC-7, Ken Barber wrote:

> 1. How do I address the problem of forge vs docs site redundancy? Not sure
> that the above redmine area is the correct place

Its a good enough place to start. If the documentation overlaps or
says two different things its a fair place to cover it.

Okay, I'll keep that in mind for future reference.

> 2. The trouble with the bug I filed, though.. is that personally, I think
> the documentation in that area could do with a *Major* rewrite.
> I filed some stopgap measures, but a rewrite is called for.
> It would not make sense to do that until the "forge vs docs site" issue is
> decently resolves, though.
>
> Additionally, I do have some experience writing online documentation, and
> could potentially contribute.
> The bug system is not adequate for doing that, though.

On that URL I gave you:

http://projects.puppetlabs.com/projects/puppet-docs

There is a link on how to contribute:

http://docs.puppetlabs.com/contribute.html

Funily enough, I just read that.
It mentions using git, and remote branches, for direct edits... but it does not mention how to actually *view* your  changes in a webbrowser, that I can see.
(that is to say, viewing how the complete page will look after your edit)

It only seems to mention
(fork it)
(make your edits)
(commit)
(submit redmine bug with branch url)

[Ken Barber]

Its managed like code, so if you go to:

https://github.com/puppetlabs/puppet-docs

You can see the README file with the docs on how to build it, and view it.

ken.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-dev/-/LqISOhM9i7AJ.
>
> To post to this group, send email to puppe...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-dev+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-dev?hl=en.

[Philip Brown]

On Thursday, May 10, 2012 12:31:15 PM UTC-7, Ken Barber wrote:

Its managed like code, so if you go to:

https://github.com/puppetlabs/puppet-docs

You can see the README file with the docs on how to build it, and view it.

Wow.
(asking for clarity here, so I can submit a change suggestion for  http://docs.puppetlabs.com/contribute.html that is accurate)

So, if one wishes to directly edit, what happens, essentially, is

1. you check out the full doc tree from git
2. you start up some custom ruby/rake server locally, the code for which is also in "the full doc tree from git", that mimics your own personal docs.puppetlabs.com
3. you view & edit your local copy as desired
4. submit git changes when happy
?

[Ken Barber]

Yes - I followed the documentation on the github README and it seemed
to work as documented, the only thing I needed to do extra was include
the rdoc gem beforehand.

I'm on a Mac - so your personal mileage may vary depending on the OS
you are trying to use.

ken.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/puppet-dev/-/91RuxCtjbb0J.