Dependacies with Custom Types

75 views
Skip to first unread message

apgrif...@gmail.com

unread,
Jul 11, 2016, 10:06:00 PM7/11/16
to Puppet Users
Hi,

I have created a custom type which uses the ipaddress rubygem in the validation function. However, this package is not part of my standard build and I can find no way of installing it from Puppet as the catalog compilation fails with "Could not autoload type".

Is there anyway to manage this type of dependency within Puppet, or should I look to do it outside?

Puppet version: 3.6

Thanks,

Alan

Rob Nelson

unread,
Jul 11, 2016, 11:07:43 PM7/11/16
to puppet...@googlegroups.com
I don't know the specifics of how it works, but the puppet/zabbix module has a type that relies on the zabbixapi which it also provides. You may want to review that module to see how the package and type are implemented in detail. 
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/845debad-5c81-41b8-b3c9-a1a1405ef3b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--

apgrif...@gmail.com

unread,
Jul 12, 2016, 8:09:04 AM7/12/16
to Puppet Users
I had a look at the Zabbix module. I think it works there as the dependency is in the provider rather than in the type. Having thought about some more I think it's easier to just remove the dependency on that module.


On Tuesday, 12 July 2016 00:07:43 UTC+1, Rob Nelson wrote:
I don't know the specifics of how it works, but the puppet/zabbix module has a type that relies on the zabbixapi which it also provides. You may want to review that module to see how the package and type are implemented in detail. 

On Monday, July 11, 2016, <apgrif...@gmail.com> wrote:
Hi,

I have created a custom type which uses the ipaddress rubygem in the validation function. However, this package is not part of my standard build and I can find no way of installing it from Puppet as the catalog compilation fails with "Could not autoload type".

Is there anyway to manage this type of dependency within Puppet, or should I look to do it outside?

Puppet version: 3.6

Thanks,

Alan

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.


--

Josh Cooper

unread,
Jul 13, 2016, 5:52:58 AM7/13/16
to puppet...@googlegroups.com
On Tue, Jul 12, 2016 at 1:09 AM, <apgrif...@gmail.com> wrote:
I had a look at the Zabbix module. I think it works there as the dependency is in the provider rather than in the type. Having thought about some more I think it's easier to just remove the dependency on that module.

On Tuesday, 12 July 2016 00:07:43 UTC+1, Rob Nelson wrote:
I don't know the specifics of how it works, but the puppet/zabbix module has a type that relies on the zabbixapi which it also provides. You may want to review that module to see how the package and type are implemented in detail. 

On Monday, July 11, 2016, <apgrif...@gmail.com> wrote:
Hi,

I have created a custom type which uses the ipaddress rubygem in the validation function. However, this package is not part of my standard build and I can find no way of installing it from Puppet as the catalog compilation fails with "Could not autoload type".

If you have a type with a dependency on a gem and the autoloader fails to load the type, then it often times means either the type (or something it is transitively loading) is directly requiring a gem. That is a puppet anti-pattern, because it means puppet may take two runs to converge, once to install the gem, and again to sync the resource whose type/provider depends on the gem.

For example, don't do this:

  require 'ipaddress'
  
  Puppet::Type.type(:mytype) do
    newproperty(:address) do
      validate do |value|
        raise "..." unless IPAddress.valid_ipv4?(value)
      end
    end
  end

The above code has two problems. First, the require statement is evaluated on the master during compilation(!), and the gem may very well not be installed there, especially if the gem is OS/platform-specific, e.g. win32-dir.

Second, the `validate` block is only called on the agent when the agent converts a server catalog (JSON) into an agent catalog (RAL). So the above code is requiring the ipaddress gem on the master, but never using it. To ensure the require statement is only evaluated on the agent, you could instead do:

  validate do |value|
    require 'ipaddress'
    raise "..." unless IPAddress.valid_ipv4?(value)
  end

However, there's no guarantee puppet will have installed the ipaddress gem prior to applying resources for your custom type (unless you add an explicit before/require dependency). Also due to rubygems caching, the newly installed gem may not be visible, so the require statement may fail, but magically work the next time puppet runs,

The idiomatic way to handle this in puppet is to create a provider feature, and confine the provider based on the presence of the feature. So define a feature in your module directory lib/puppet/feature/ipaddress.rb, containing:

  Puppet.features.add(:ipaddress, :libs => ['ipaddress'])

And confine your provider based on the presence of the feature:

  Puppet::Type.type(:mytype).provide(:myprovider) do
    confine :feature => :ipaddress
  end

Have the type call the provider to validate the address, or add the validation logic to the provider's setter method for that property, e.g. address=.

  validate do |value|
    provider.validate_address(value)
  end

If you structure your module this way then you can create a manifest like the following, and puppet will always "do the right thing", ensuring the ipaddress gem is installed prior to evaluating the mytype resource, and that will work within a single puppet run:

  mytype { 'foo':
    address  => '12.34.56.78'
  }
  package { 'ipaddress':
    ensure   => installed,
    provider => gem
  }

Here are some other examples where we've fixed custom types/providers to "install all the things" in a single run:


Is there anyway to manage this type of dependency within Puppet, or should I look to do it outside?

Puppet version: 3.6

Thanks,

Alan

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.


--

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/ff193af2-9a75-4930-afca-ce2ca87779b4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Josh

--
Josh Cooper
Developer, Puppet

PuppetConf 2016, 19 - 21 October, San Diego, California
Early Birds save $350 - Register by 15 July

jcbollinger

unread,
Jul 13, 2016, 12:51:49 PM7/13/16
to Puppet Users


On Wednesday, July 13, 2016 at 12:52:58 AM UTC-5, Josh Cooper wrote:
[A clear and detailed explanation...]

That was great, Josh.  Are the same instructions available somewhere among Puppet's online docs and tutorials?  If not, they should be.

John

Rob Nelson

unread,
Jul 13, 2016, 1:54:23 PM7/13/16
to puppet...@googlegroups.com
I second this motion, that was a great explanation. I'd love to see that documented well and maybe in a git repo with a 'bad' and 'good' branch to better examine the differences, if at all possible!


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/774fe4a8-0aa4-4227-958a-b97e5e3afe75%40googlegroups.com.

Josh Cooper

unread,
Aug 11, 2016, 10:35:39 PM8/11/16
to Puppet Users
Thanks Rob and John. I filed to https://tickets.puppetlabs.com/browse/DOCUMENT-573 to improve the docs in this area. Feel free to comment on the ticket.
Reply all
Reply to author
Forward
0 new messages