Custom types in environments working .... yes/no/maybe ?

[Douglas Garstang]

My issue may be related to this bug:

http://projects.puppetlabs.com/issues/13858

"Custom types in environments require loading into master's libdir"

However, now I'm not so sure, This was working previously with a given
client. However, after trying on a fresh client, it's failing with:

err: Could not run Puppet configuration client: Could not find a
default provider for logical_volume

The server seems to have the files:
/var/lib/puppet/lib/puppet/provider/logical_volume
/var/lib/puppet/lib/puppet/type/logical_volume.rb

And so does the client:
/var/lib/puppet/lib/puppet/type/logical_volume.rb
/var/lib/puppet/lib/puppet/provider/logical_volume

pluginsync=true in /etc/puppet/puppet.conf. Using multiple
environments and puppet 2.7.1.

*sigh*

Doug.

[Douglas Garstang]

I tried applying the patch at
https://github.com/puppetlabs/puppet/commit/96712efeb543928704fc9938e7429552d8ded039
on both the server and client. That had the effect of changing the
error on the client from:

err: Could not run Puppet configuration client: Could not find a
default provider for logical_volume

to:

err: Could not run Puppet configuration client: Could not find a

default provider for volume_group

Doug.

[Douglas Garstang]

What a horrible clusterf*ck mess.

http://www.mailinglistarchive.com/html/puppet...@googlegroups.com/2010-03/msg00801.html

Doug.

On Wed, Aug 15, 2012 at 10:41 AM, Douglas Garstang

--
Regards,

Douglas Garstang
http://www.linkedin.com/in/garstang
Email: doug.g...@gmail.com
Cell: +1-805-340-5627

[Daniele Sluijters]

Hi,

We've ran into the same problem. Our current solution, which we're not happy about, is to also have the custom types on the master/production environment. Doing that at least the custom types are always available. Unfortunately, it makes it impossible to deploy an update of a custom type to a branch/environment for testing.

-- 

Daniele Sluijters

[jcbollinger]

On Thursday, August 16, 2012 3:26:23 AM UTC-5, Daniele Sluijters wrote:

Hi,

We've ran into the same problem. Our current solution, which we're not happy about, is to also have the custom types on the master/production environment. Doing that at least the custom types are always available. Unfortunately, it makes it impossible to deploy an update of a custom type to a branch/environment for testing.

For what it's worth, I would recommend using an entirely separate Puppet process for testing.  If you like, it can run on the same server, but a different port.  That would have several advantages for you, among them

• you would be able to test modifications to your Puppet extensions without putting them into production
• you would be able to test how changes apply to and affect your various environments (or if you use environments only for this purpose, then you could drop them altogether)
  
• your production master would not be exposed to any unproven code
• it would fit in more intuitively (to me) with source control, branches, etc.

John

[Daniele Sluijters]

I'm afraid I don't completely agree there. One of the benefits of environments is that it allows you to completely separate a change for a module/extension into a separate environment, without needing a separate master, SSL CA and so forth.

As far as the point to not exposing the production master to untested code, fair enough. I guess there's a chance that untested Ruby code in a different environment could crash the master. Then again, I'll probably notice that about 10 seconds after deploying the actual code and if an agent can't check in for a few minutes that's not necessarily a problem.

The point that we would be able to test Puppet extensions without putting them into production is moot really if Puppet would actually bother to load extensions from a environment specific libdir. There's plenty of ways to work around the fact that extensions in environments don't really work as one would expect them to but, that really ought be fixed in the first place, or environments removed all together. In it's current state the behaviour is slightly unspecified.

The patch suggested in the ticket seems reasonable enough though I'm guessing one would have to tell Puppet its libdir now looks like /var/lib/puppet/master/lib/<%= environment %> for it to work.

[Jakov Sosic]

On 08/16/2012 10:38 PM, Daniele Sluijters wrote:

> The point that we would be able to test Puppet extensions without
> putting them into production is moot really if Puppet would actually
> bother to load extensions from a environment specific libdir. There's
> plenty of ways to work around the fact that extensions in environments
> don't really work as one would expect them to but, that really ought be
> fixed in the first place, or environments removed all together. In it's
> current state the behaviour is slightly unspecified.

+1


--
Jakov Sosic
www.srce.unizg.hr

[jcbollinger]

On Thursday, August 16, 2012 3:38:10 PM UTC-5, Daniele Sluijters wrote:

I'm afraid I don't completely agree there. One of the benefits of environments is that it allows you to completely separate a change for a module/extension into a separate environment, without needing a separate master, SSL CA and so forth.

I'm sorry, disagreeing with me is not permitted.  :-)

Seriously, though, if you use environments only for separating your production code from your test code and maybe from your development code then -- to the extent that works for you -- fine.  I wouldn't do it that way, but you're not me (for which I'm sure a great many people are thankful).

If you are also using environments for other purposes, however, such as different environments for different clients, different departments, or the like, then I think you really lose something, or at least incur a lot higher cost, by simultaneously using that same dimension to provide for testing.  Note in particular that a separate master does not necessarily have to have separate CA, certificate, etc.; in fact, if the masters are colocated on the same host then it should be pretty easy to make them share.
 

As far as the point to not exposing the production master to untested code, fair enough. I guess there's a chance that untested Ruby code in a different environment could crash the master. Then again, I'll probably notice that about 10 seconds after deploying the actual code and if an agent can't check in for a few minutes that's not necessarily a problem.

Remember that we're working in Ruby, so code the master loads can dynamically modify code that's already loaded and running (subject to some constraints).  If I actually wanted to crash the master via a custom extension, then I'm pretty confident that I could do it.  I don't have any doubt that a buggy extension could do it accidentally.  Worse than outright crashes, however, would be subtler misbehaviors that you might not recognize immediately.  Depending on the nature of the misbehavior, a lot of damage could be caused.

 

The point that we would be able to test Puppet extensions without putting them into production is moot really if Puppet would actually bother to load extensions from a environment specific libdir. There's plenty of ways to work around the fact that extensions in environments don't really work as one would expect them to but, that really ought be fixed in the first place, or environments removed all together. In it's current state the behaviour is slightly unspecified.

The patch suggested in the ticket seems reasonable enough though I'm guessing one would have to tell Puppet its libdir now looks like /var/lib/puppet/master/lib/<%= environment %> for it to work.

I haven't examined the patch, but my main concern would be about related issue http://projects.puppetlabs.com/issues/12173, which concerns different versions of the same extension being relied upon by different environments (it doesn't work), and which is explicitly not addressed by the proposed patch for issue 13858.  Where different environments want different versions of extensions, such as if you're trying to use environments to provide a testbed for extension development or for third-party module testing, I would think 12173 would be a blocker.


John