Puppet Custom Types, the easy way

[bert hajee]

We have written a library to make it easier to build Puppet Custom Types. Check out our first blog post introducing easy_type. I'm really curious if this fits the needs of any Puppet developers out there. Please let me know if you like it. Also let me know if you don't like it or would like to see some changes.

Regards,

Bert hajee

[Luke Kanies]

From: bert hajee bert hajee
Reply: puppe...@googlegroups.com puppe...@googlegroups.com
Date: January 26, 2014 at 10:27:04 AM
To: puppe...@googlegroups.com puppe...@googlegroups.com
Subject:  [Puppet-dev] Puppet Custom Types, the easy way

We have written a library to make it easier to build Puppet Custom Types. Check out our first blog post introducing easy_type. I'm really curious if this fits the needs of any Puppet developers out there. Please let me know if you like it. Also let me know if you don't like it or would like to see some changes.

Hi Bert,

I love the idea of a library that makes it much easier to build native types and providers.

Am I reading correctly that most of the work is in the type, thought?  Is it adding the right getters and setters on the provider?

-- 
http://puppetlabs.com | http://about.me/lak | @puppetmasterd

[bert hajee]

Luke,

 

Am I reading correctly that most of the work is in the type, thought?  Is it adding the right getters and setters on the provider?

Yes it does. Actually it's more like a type on steroids and adding the information the provider needs. This way it hides most of the gory details from the provider. So using easy_type, most of the work is done in the type and the provider becomes standard. 

Here's an example of a type managing a role in an oracle database:

module Puppet

  newtype(:role) do

    include EasyType

    include ::Utils::OracleAccess

    desc "This resource allows you to manage a role in an Oracle database."

    set_command(:sql)

    ensurable

    to_get_raw_resources do

      sql "select * from dba_roles"

    end

    on_create do

    "create role #{self[:name]}"

    end

    on_modify do

      "alter role#{self[:name]}"

    end

    on_destroy do

      "drop role#{self[:name]}"

    end

    parameter :name

    property  :password

  end

end

The parameter and the property directive's sort of include a file. Here's the content of the parameter file:

newparam(:name) do

  include EasyType

  include EasyType::Validators::Name

  include EasyType::Mungers::Upcase

  desc "The role name "

  isnamevar

  to_translate_to_resource do | raw_resource|

  raw_resource.column_data('ROLE').upcase

  end

end

You can check the github repository for some Oracle types  to see some of the more difficult types. Obviously it overrides the separation of concerns between the type and the provider. But we noticed that for types inside an Oracle database or some other middleware, there is hardly ever the need for the indirection, the provider gives us.

Love to hear your opinion about what we did. We are open to any suggestions you might have.

Regards,

Bert Hajee

 

[bert hajee]

Just reread my reaction. Maybe it's not clear what a provider looks like for an easy_type. To make it really clear, here is the provider:

require 'easy_type'

require 'utils/oracle_access'

Puppet::Type.type(:oracle_user).provide(:simple) do

include EasyType::Provider

  desc "Manage Oracle users in an Oracle Database via regular SQL"

  mk_resource_methods

end

Besides the description. It's actually the same for ant easy_type.

Regards,

Bert Hajee

[Luke Kanies]

From: bert hajee bert hajee
Reply: puppe...@googlegroups.com puppe...@googlegroups.com

Date: January 28, 2014 at 10:44:42 PM
To: puppe...@googlegroups.com puppe...@googlegroups.com
Subject:  Re: [Puppet-dev] Puppet Custom Types, the easy way

Yeah, I looked at the provider.  My question is whether that provided gets modified by all the methods in easy_type, or whether it stays like it is, and all the work is done in the type.

We used to not have providers, and instead all work was done in types, but now ideally the provider is the class that runs the shell scripts, talks to the system, etc., and the type is just responsible for modeling the resource, doing data validation, etc.

[Henrik Lindberg]

On 2014-31-01 1:22, Luke Kanies wrote:
> From: bert hajee bert hajee <mailto:bert....@gmail.com>
> Reply: puppe...@googlegroups.com puppe...@googlegroups.com
> <mailto:puppe...@googlegroups.com>

> Date: January 28, 2014 at 10:44:42 PM
> To: puppe...@googlegroups.com puppe...@googlegroups.com

> <mailto:puppe...@googlegroups.com>

> Subject: Re: [Puppet-dev] Puppet Custom Types, the easy way
>> Just reread my reaction. Maybe it's not clear what a provider looks
>> like for an easy_type. To make it really clear, here is the provider:
>>
>> require 'easy_type'
>> require 'utils/oracle_access'
>>
>> Puppet::Type.type(:oracle_user).provide(:simple) do
>> include EasyType::Provider
>>
>> desc "Manage Oracle users in an Oracle Database via regular SQL"
>>
>> mk_resource_methods
>>
>> end
>>
>> Besides the description. It's actually the same for ant easy_type.
>
> Yeah, I looked at the provider. My question is whether that provided
> gets modified by all the methods in easy_type, or whether it stays like
> it is, and all the work is done in the type.
>
> We used to not have providers, and instead all work was done in types,
> but now ideally the provider is the class that runs the shell scripts,
> talks to the system, etc., and the type is just responsible for modeling
> the resource, doing data validation, etc.
>

I can imagine the system evolving even further in that direction and
ultimately requiring that any behavior above basic data type / integrity
checks is implemented in a provider.

Regards
- henrik

[Trevor Vaughan]

+100 for Type/Provider separation.

I remember the good old days quite well and was really happy when that got split out.

Trevor

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/etPan.52eaecbc.3d00b9d9.20af%40claymore.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[bert hajee]

Op zaterdag 1 februari 2014 02:33:41 UTC+1 schreef Trevor Vaughan:

+100 for Type/Provider separation.

I remember the good old days quite well and was really happy when that got split out.

Trevor

The discussion seems to go in the direction of not having a provider. easy_type is by no means at all meant to stop using a provider. Actually, it is by no means a total replacement for the Type and Provider pattern. What it is, is an add-on module for people, less knowledgeable about Puppet (and the intricacies of Types and providers) to have a custom type for an easy kind of resource. Probably only on one kind of OS. 

 

That said about the current discussion, what I like about the discussion, is that we seem to all have the idea, it should be easier to make custom types. While we disagree on (at least some) of the directions I took with easy_type. Why not turn this into a discussion on what we would like to see in an easy way to build a type.

I'l start, but please join:

- A way to keep it close to people who know about the resource, but less about Puppet

- Be able to easy abstract different OS-es and version (Like the current provider

- Make it easier to call a set of standard available (and extendable) validators

- Make it easier to call a set of standard available (and extendable) mungers.

- Make it easy to gradually build the os command by stating a base part and extending it if a property or a name is specified in the resource definition.

Bert

[bert hajee]

Op vrijdag 31 januari 2014 01:22:20 UTC+1 schreef Luke Kanies:

The provider uses callbacks to Type to do the work. In the Type, you define on_create, on_modify and on_delete methods that take a block. The block returns the basic os command you need to do the work. The property also has a callback method, namely the on_apply. The return value of the on_apply method for any modified property is append to the base command.

An example:

In the type:

do_command(:sql)

on_modify do

  "alter tablespace #{resource[:name]}"

end

in the property definition for size

on_apply do

  "set size #{resource[:size]}"

end

When the provider notices the size in the manifest is different then the actual size, it builds the command:

sql alter tablespace  set size 10K

--- ----------------  ------------

 1               part 2                           part 3

part 1 is from the command

part 2 is from the on_modify

part 3 is from the property definition

[Johan De Wit]

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/a1c9c311-a409-47ca-9467-4097f81729f9%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

just a sidenote.  Where does the testing of the code fits in ?

grts

johan 

-- 
Johan De Wit

Open Source Consultant

Red Hat Certified Engineer         (805008667232363)
Puppet Certified Professional 2013 (PCP0000006)
_________________________________________________________
 
Open-Future                 Phone     +32 (0)2/255 70 70
Zavelstraat 72              Fax       +32 (0)2/255 70 71
3071 KORTENBERG             Mobile    +32 (0)474/42 40 73
BELGIUM                     http://www.open-future.be
_________________________________________________________
 

Upcoming Events:

Puppet Fundamentals Training | http://www.open-future.be/puppet-fundamentals-training-4-till-6th-february

Puppet Introduction Course | http://www.open-future.be/puppet-introduction-course-7th-february

Zabbix Certified Training | http://www.open-future.be/zabbix-certified-training-10-till-12th-february

Zabbix for Large Environments Training | http://www.open-future.be/zabbix-large-environments-training-13-till-14th-february

Subscribe to our newsletter: http://eepurl.com/BUG8H

[John Bollinger]

On Thursday, January 30, 2014 6:27:08 PM UTC-6, henrik lindberg wrote:

I can imagine the system evolving even further in that direction and
ultimately requiring that any behavior above basic data type / integrity
checks is implemented in a provider.

Types do provide a handy place to put code that all providers need.  Especially so if the shared code is characteristic of the modeled resource type itself, as opposed to being associated with a particular implementation of that resource.

Generally speaking, though, I quite agree that type / provider separation is a good thing, and that the bias should be to move further in that direction.


John

[John Bollinger]

On Saturday, February 1, 2014 7:48:53 AM UTC-6, bert hajee wrote:

The discussion seems to go in the direction of not having a provider.

I must have missed all that.

 

easy_type is by no means at all meant to stop using a provider. Actually, it is by no means a total replacement for the Type and Provider pattern. What it is, is an add-on module for people, less knowledgeable about Puppet (and the intricacies of Types and providers) to have a custom type for an easy kind of resource.

And as far as it goes, it looks like a good and useful tool, as long as its limits are understood.

 

Probably only on one kind of OS. 

 

I think that's an understatement.  Inasmuch as the key methods are defined on the type, the module really supports only custom types with a single provider.  That is to say, although additional providers could be written, it looks like easy_type would start to get in the way more than it helps for the second and subsequent provider.  That's not necessarily a problem in any particular case, but it does make easy_type a special-purpose tool.

 

That said about the current discussion, what I like about the discussion, is that we seem to all have the idea, it should be easier to make custom types. While we disagree on (at least some) of the directions I took with easy_type. Why not turn this into a discussion on what we would like to see in an easy way to build a type.

I'l start, but please join:

- A way to keep it close to people who know about the resource, but less about Puppet

- Be able to easy abstract different OS-es and version (Like the current provider

- Make it easier to call a set of standard available (and extendable) validators

- Make it easier to call a set of standard available (and extendable) mungers.

- Make it easy to gradually build the os command by stating a base part and extending it if a property or a name is specified in the resource definition.

Absolute top of my list, ever since I first played with custom types:
- Formal, complete, public documentation of the API Puppet provides to custom types and providers
- Formal, complete, public documentation of the API Puppet expects custom types and providers to expose

Nothing else even comes close.  The guides PL provides around this area are helpful -- and more so now than they used to be -- but they are not an adequate substitute for bona fide API docs.  Moreover, without good reference docs it's hard even to discuss what new features could benefit Puppet, because there is no shared understanding of what Puppet already has.


John

[Luke Kanies]

From: John Bollinger John Bollinger
Reply: puppe...@googlegroups.com puppe...@googlegroups.com
Date: February 3, 2014 at 4:04:57 PM

To: puppe...@googlegroups.com puppe...@googlegroups.com
Subject:  Re: [Puppet-dev] Puppet Custom Types, the easy way

On Saturday, February 1, 2014 7:48:53 AM UTC-6, bert hajee wrote:

The discussion seems to go in the direction of not having a provider.

I must have missed all that.

Yeah, I think that would be a bad overall direction, both because it reduces flexibility, and also because over time Puppet will switch to requiring providers.

Hopefully EricS can chime in here, but I know API docs are something the team is working on pretty heavily.

The main problem in this area is that there’s quite a bit of stuff that also needs to be deprecated, so the balance is between fixing things vs documenting them.