Delving into Puppet code -- where should I ask?

[Victor Danilchenko]

So, as I posted on the adjacent thread, I have been having a problem with Puppet master 3.7 not reading the contents of my directory environments; it recognizes their existence, but ignores modules, manifests, and environment.conf therein.

Trying to fish this problem out, I decided to go some code spelunking, and I quickly realized that tracing my way through that code is very hard -- the execution paths are weird and twisted. I have no idea where stuff is SUPPOSED to happen; I can see that my environment/production/modules directory isn't getting read, but I don't know where it's supposed to get read.

So, I was thinking about finding some puppet developers, and simply asking for some pointers -- namely, in which module/class/method is Puppet supposed to actually scan environment contents? I think I can take it from there, and figure our why my environment modules directory gets ignored.

A quick google search turned up Puppet Developers group; however, I have a suspicion that such a question might be inappropriate there -- it's for actual puppet coding, not use and troubleshooting.

So, can anyone suggest a useful venue where I could ask questions related to Puppet source code?

Thanks.

[Riley Shott]

Hi Victor,

Have you verified that you have the necessary options specified in your puppet.conf to turn on directory environments?

https://docs.puppetlabs.com/puppet/latest/reference/environments_configuring.html#enabling-directory-environments-in-puppet-enterprise

https://docs.puppetlabs.com/puppet/latest/reference/environments_configuring.html#enabling-directory-environments-in-open-source-puppet

Below is a snippet of what we specified (open source Puppet 3.7.0).

[master]
default_manifest = ./manifests
environmentpath = $confdir/environments

$confdir/environments expands out to be '/etc/puppet/environments'.

-Riley

---

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5bdf2b86-f716-4921-a3b5-e7977947fb2b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Victor Danilchenko]

Yes, I have. The salient portion of my puppet.conf file is:

[main]
 environmentpath = $confdir/environments
 basemodulepath = $confdir/modules:/usr/share/puppet/modules

Puppet recognizes my environments (I only have 'production'), because when I supply an invalid environment name, it complains; but though my environment is correctly populated with modules and manifests (I simply copied it wholesale from our operational production Puppet 3.4 master, so I know it's correct), the command 'puppet module list' only searches /usr/share/puppet/modules, and returns the 'no modules installed' message. The environments/production/environment.conf file apparently gets ignored as well; at least overriding modulepath in it doesn't work.

The gory details are in the adjacent thread.

So, since it seems nobody can see what's going wrong, I wanted to try to trace the code and find where the error occurs; except that finding the place where something isn't happening, is quite hard. I could use some code pointers...

[jcbollinger]

If you have questions specifically about how Puppet's source code is organized and / or how it actually works, then the Puppet Developers group seems a fine place for it.  That the reason for your interest is related to troubleshooting your installation is not much relevant, I think.  You're already far ahead in having considered in the first place whether that's a relevant forum.


John

[Riley Shott]

Hm. I wonder if this is related, but I suspect if it was you would get complaints instead of catalog runs that succeed, albeit empty. I'll mention it anyway because you will come across this eventually.

We've had to symlink modules with custom types and providers in all environments to /etc/puppet/modules so that pluginsync will work. AFAIK this is a issue with Ruby's containment and namespace capabilities (part of the reason they're re-writing Puppet 4 in Clojure). Your master's /var/lib/puppet/lib is where is its agent stores all the plugins, and it's also where it distributes them from. What this means is that the Puppet master needs to run in order to sync any new plugins before it can distribute them.

Your config look correct now since you've changed default_manifests to './manifests'. Try changing the modulepath value in your environment.conf to this:

modulepath = modules

-Riley

---

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/96645f88-e8e2-4f8c-9366-2a8d999c5e06%40googlegroups.com.

[Riley Shott]

As for other places to ask. You can the Puppet channels on the freenode IRC network:

http://projects.puppetlabs.com/projects/1/wiki/Irc_Channel

#puppet

#puppet-dev

-Riley

---

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1073577765.52616241.1414526431538.JavaMail.zimbra%40sfu.ca.

[Victor Danilchenko]

When I change modulepath to 'modules' in environment.conf, I still get the 'no modules installed' error. :(

[jcbollinger]

On Tuesday, October 28, 2014 3:00:38 PM UTC-5, Riley Shott wrote:

Hm. I wonder if this is related, but I suspect if it was you would get complaints instead of catalog runs that succeed, albeit empty.

That catalog runs are succeeding but yielding empty catalogs is a good reason to consider access control issues.

We've had to symlink modules with custom types and providers in all environments to /etc/puppet/modules so that pluginsync will work.

That sounds strange.  Is there a bug report on file for this?  If not then you should file one.

 

AFAIK this is a issue with Ruby's containment and namespace capabilities (part of the reason they're re-writing Puppet 4 in Clojure).

Those details really ought not to be much relevant to pluginsync.  There are namespace and containment issues, but as far as I know they relate to which plugin components the master will load, itself, not which ones it will sync.

 

Your master's /var/lib/puppet/lib is where is its agent stores all the plugins, and it's also where it distributes them from.

To the best of my knowledge, only the first part (that /var/lib/puppet/lib is where pluginsync drops plugins) is strictly true.  I am inclined to think that only the first part is true at all.

 

What this means is that the Puppet master needs to run in order to sync any new plugins before it can distribute them.

Let's save this for when it's relevant.  Victor needs to get environments working at his master before any of those details begin to matter.

 

Your config look correct now since you've changed default_manifests to './manifests'.

The default_manifests setting is a red herring, given that Victor has a 'manifests' directory in each environment.  Moreover, the setting he started with is the one recommended by the documentation.  './manifests' is the default value for the setting, specifying that is equivalent to not specifying the property at all (which is in fact a reasonable thing to do).

 

Try changing the modulepath value in your environment.conf to this:

I think Victor said he would prefer not to use environment.conf files, and as long as he uses the standard environment layout he shouldn't need to do.  Better to keep environment.conf out of the picture entirely until basic function is achieved.  In fact, better to keep modulepath out of the picture until basic function is achieved, too.  It should be enough to focus on the manifest directory of one environment, and to expand from there once that's sorted.


John

[Riley Shott]

There is a bug report open, which does talk about Puppet serving plugins from /var/lib/puppet/lib.

https://tickets.puppetlabs.com/browse/PUP-731

John, you can ping me off-list if you'd like as we're going off-topic, but are you saying that you're successfully using pluginsync with directory environments, and without the need to symlink these types of modules into a common module path (i.e. one your Puppet masters include/look at)?

And as for the pluginsync issue, you're right. It relates to which plugins they'll load, my apologies for mixing it up, but it doesn't really matter in the end; it's broken.

-Riley

---

From: "jcbollinger" <John.Bo...@stJude.org>
To: puppet...@googlegroups.com
Sent: Tuesday, 28 October, 2014 14:25:04
Subject: Re: [Puppet Users] Delving into Puppet code -- where should I ask?

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/ece2092b-9f12-4016-b21a-5439dee7ef35%40googlegroups.com.