Issue with module from forge - Duplicate declaration Class[Rsyslog]

279 views
Skip to first unread message

re-g...@wiu.edu

unread,
Oct 5, 2016, 3:32:37 PM10/5/16
to Puppet Users
I installed the puppet module saz-rsyslog from puppet forge.
I use The Foreman to configure nodes. The Foreman is used by puppet via configuration [master] "external_nodes" "/etc/puppet/node.rb"

Since the saz-rsyslog module install, I have been receiving the following error off and on (not consistently) across many nodes on a puppet update (i.e. puppet agent -t):

"Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Rsyslog] is already declared; cannot redeclare on node <node-name>"


My nodes are CentOS 5,6,7; and any various number of the nodes may experience this issue, but not all of them at the same time.

One day I will see dozens of server with this error, and other nodes not having this issue. This may go on for days if I do not touch The Foreman.
I'll make some changes to host configuration for puppet module class parameters in The Foreman - never the saz-rsyslog module though..
After the changes, half or more of the servers having issue (not all) will magically have no problems.
However, more nodes that did not have issues before, will now experience this issue.

Also, this change of events is not directly related to The Foreman host configuration changes.
I can simply perform a puppet module upgrade to a unrelated module (e.g. mine-yumconfig). After upgrading the unrelated module, again many nodes with this issue will now have it resolved, and different ones not experiencing the issue before will now begin experiencing it.


"... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..."


Can anyone help me understand this issue, or help me get it resolved permanently?

When I search for answers, all I see are "You have written a duplicate class in your module." However, in my case, I did not write the saz-rsyslog module, I am only using it. It is a puppet-forge approved module with 635,000+ downloads. And without modifying the module, the issue can disappear, seemingly without rhyme or reason.

-RG

Rob Nelson

unread,
Oct 5, 2016, 4:58:38 PM10/5/16
to puppet...@googlegroups.com
RG, typically the Duplication declaration message is followed by some indicator of the file where it is duplicated and the line number. Are you receiving that information and maybe snipped it off the end of the message you included? An alternative might be any sort of `hiera_include()` or `create_resources()` calls whose arguments may include `rsyslog` more than once.


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1ead58c4-ed05-4b03-9bee-6fd576d187f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jcbollinger

unread,
Oct 6, 2016, 9:27:40 AM10/6/16
to Puppet Users


On Wednesday, October 5, 2016 at 2:32:37 PM UTC-5, re-g...@wiu.edu wrote:
I installed the puppet module saz-rsyslog from puppet forge.
I use The Foreman to configure nodes. The Foreman is used by puppet via configuration [master] "external_nodes" "/etc/puppet/node.rb"

Since the saz-rsyslog module install, I have been receiving the following error off and on (not consistently) across many nodes on a puppet update (i.e. puppet agent -t):

"Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Rsyslog] is already declared; cannot redeclare on node <node-name>"



It seems highly unlikely that merely installing the module on the master would cause any catalog compilations to fail, and certainly the error message indicates that there are some declarations of its classes in play.  It is not the mere presence of the module, but rather the declarations of one or more classes in it that is causing you trouble.

 

My nodes are CentOS 5,6,7; and any various number of the nodes may experience this issue, but not all of them at the same time.

One day I will see dozens of server with this error, and other nodes not having this issue. This may go on for days if I do not touch The Foreman.
I'll make some changes to host configuration for puppet module class parameters in The Foreman - never the saz-rsyslog module though..
After the changes, half or more of the servers having issue (not all) will magically have no problems.
However, more nodes that did not have issues before, will now experience this issue.

Also, this change of events is not directly related to The Foreman host configuration changes.
I can simply perform a puppet module upgrade to a unrelated module (e.g. mine-yumconfig). After upgrading the unrelated module, again many nodes with this issue will now have it resolved, and different ones not experiencing the issue before will now begin experiencing it.


"... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..."


Can anyone help me understand this issue, or help me get it resolved permanently?

When I search for answers, all I see are "You have written a duplicate class in your module."


Any such claim is at best misleading.  The problem is not that you or anyone has written a duplicate class, it's that the class in question has been declared more than once, and at least one of those declarations uses the resource-like style.  In considering that, you should be aware that:
  • a declaration in resource-like style can include one emitted by your ENC (part of Foreman) if it uses the form that specifies class parameters explicitly, and
  • when one class inherits from another, any declaration of the derived class comprises an implicit declaration of its base class.  Furthermore,
  • when class inheritance is involved, the docs specify that any parameters of the base class must either take their default values or have their values supplied by automatic external data lookup.  In practice, that's too weak -- a separate resource-like declaration of the base class can cause trouble even if it specifies no parameter values.
I talked up class inheritance because I've examined the manifests of the latest version of the module, and there are indeed a few cases of class inheritance that could be feeding your problem.  There are also cases of ordinary declaration of class ::rsyslog by other classes in that module.  The bottom line is that you need to follow the advice from the docs with respect to class ::rsyslog by avoiding declaring it via the resource-like style, including preventing your ENC from doing so.  Any non-default parameters must be provided via automated data binding.  I'm afraid I'm not knowledgeable enough about The Foreman to tell you how to handle that end, however.


John

re-g...@wiu.edu

unread,
Oct 6, 2016, 9:44:40 AM10/6/16
to Puppet Users


On Wednesday, October 5, 2016 at 3:58:38 PM UTC-5, Rob Nelson wrote:
RG, typically the Duplication declaration message is followed by some indicator of the file where it is duplicated and the line number. Are you receiving that information and maybe snipped it off the end of the message you included? An alternative might be any sort of `hiera_include()` or `create_resources()` calls whose arguments may include `rsyslog` more than once.

Here is the complete output when running `puppet agent -t`:
[Linux ~]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Rsyslog] is already declared; cannot redeclare on node server1.mydomain.test1.example.com

Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
[Linux ~]#


On Wed, Oct 5, 2016 at 3:16 PM, <re-g...@wiu.edu> wrote:
I installed the puppet module saz-rsyslog from puppet forge.
I use The Foreman to configure nodes. The Foreman is used by puppet via configuration [master] "external_nodes" "/etc/puppet/node.rb"

Since the saz-rsyslog module install, I have been receiving the following error off and on (not consistently) across many nodes on a puppet update (i.e. puppet agent -t):

"Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Rsyslog] is already declared; cannot redeclare on node <node-name>"


My nodes are CentOS 5,6,7; and any various number of the nodes may experience this issue, but not all of them at the same time.

One day I will see dozens of server with this error, and other nodes not having this issue. This may go on for days if I do not touch The Foreman.
I'll make some changes to host configuration for puppet module class parameters in The Foreman - never the saz-rsyslog module though..
After the changes, half or more of the servers having issue (not all) will magically have no problems.
However, more nodes that did not have issues before, will now experience this issue.

Also, this change of events is not directly related to The Foreman host configuration changes.
I can simply perform a puppet module upgrade to a unrelated module (e.g. mine-yumconfig). After upgrading the unrelated module, again many nodes with this issue will now have it resolved, and different ones not experiencing the issue before will now begin experiencing it.


"... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..."


Can anyone help me understand this issue, or help me get it resolved permanently?

When I search for answers, all I see are "You have written a duplicate class in your module." However, in my case, I did not write the saz-rsyslog module, I am only using it. It is a puppet-forge approved module with 635,000+ downloads. And without modifying the module, the issue can disappear, seemingly without rhyme or reason.

-RG

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

re-g...@wiu.edu

unread,
Oct 6, 2016, 10:04:14 AM10/6/16
to Puppet Users
I understand the duplicate declaration issue - I have been reading about it for months trying to figure out this issue that keeps popping up. I am trying to figure out how it is possible I am getting a duplicate class declaration on a widely-used and approved module I have no control over (that no one else has reported any similar issue as mine), when using the forman to simply provide the values of parameters.

Maybe The Foreman is the cause of the ::rsyslog class to be declared more than once? Might you suggest this possibility? But then I think, how would it? And why do I have this issue for hours on a server, and then at some magic moment the issue just disappears? Why with two of my servers sitting 1 IP away from each other, one will have this issue, and the other will not? I consider that The Foreman is causing the problem, but then I should have the error on all hosts at the same time. The only different in the YAML between hosts is the IP addresses... every other puppet module parameter value is exactly the same.

The Foreman has parameterized class support. It allows us to manage the values of the puppet module parameters.
Old Screen Shots with explaination

When using the foreman, I am not writing any files, or declaring any classes. Instead, I manage the values of the puppet module parameters in the The Foreman WebUI, which is simply logic like 'if this host fact meets this criteria, then this puppet module parameter is set to this value'.





 

re-g...@wiu.edu

unread,
Oct 6, 2016, 10:49:50 AM10/6/16
to Puppet Users


On Wednesday, October 5, 2016 at 2:32:37 PM UTC-5, re-g...@wiu.edu wrote:
Some more information

I am using the latest version of the saz-rsyslog puppet module, version 4.0.3

As an example, I have this node called h1pa
Yesterday afternoon this node was getting the reported duplicate Class[Rsyslog] declaration error
The 12:15am update was the last report of this error 
The 12:45am update was the first clean update today
In fact, I had 0 nodes reporting this error

About 9:00am-ish we added a subnet and hostgroup to The Foreman. However we have not added any new nodes, nor changed the configuration to any existing nodes.

Then, I started getting the error again
The 9:15am update was the first report of this error this late morning
The 9:45am update reported this error again
My nodes reporting an error of this duplicate Class[Rsyslog] error increased from 0 to 12.

All node reports with this error are similar to h1pa node's reports.
I am seeing that many of the hosts experiencing this issue yesterday, are now experiencing it again.

-RG






 

Rob Nelson

unread,
Oct 6, 2016, 1:23:40 PM10/6/16
to puppet...@googlegroups.com
Can you undo the change in foreman, see if the problem goes away, then reimplement the change and see if the problem comes back? That would go a long way toward isolating the cause. 
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a81931c8-14b3-46d8-94ba-1f31c4c4453a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--

re-g...@wiu.edu

unread,
Oct 7, 2016, 9:48:43 AM10/7/16
to Puppet Users
On Thursday, October 6, 2016 at 12:23:40 PM UTC-5, Rob Nelson wrote:
Can you undo the change in foreman, see if the problem goes away, then reimplement the change and see if the problem comes back? That would go a long way toward isolating the cause. 


So I removed the rsyslog configuration from the foreman, the problem disappeared, then I added the rsyslog configuration back in and the problem immediately reappeared.

The other configuration we added at 9:00am-ish .... well we since added some very minor configuration changes for host parameters in the afternoon, and this problem went away. Nothing was touched after that. Then about 6 hours ago, at 2:00am-ish, the problem reappeared.
Then I removed the rsyslog configuration, after which the problem disappeared, then I re-added the rsyslog configuration and the problem reappeared.

Rob Nelson

unread,
Oct 7, 2016, 9:52:20 AM10/7/16
to puppet...@googlegroups.com
Not being a foreman user, I can't add a whole lot of detail to this, but it does SEEM like there's something in the foreman "extras" that may be conflicting with the direct puppet code in certain situations. My understanding is that it provides some sort of overlay, and that could definitely cause a conflict - it's easy with PE, for instance, to have the Node Classifier and your puppet manifests on disk have collisions without being obvious. I would look at what the "overlay" provides - whatever the proper term is for it.


To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c5246478-278c-4d2b-ac0b-4e8986a45611%40googlegroups.com.

re-g...@wiu.edu

unread,
Oct 7, 2016, 11:39:58 AM10/7/16
to Puppet Users
Well... The node I have been testing the duplicate declaration on uses a puppet secondary-master server, as it is on a remote network segment. It does not connect directly to the puppet primary-master in which The Forman is running on.

So I did some work to get this particular "server1" node to use the puppet primary-master that The Foreman is running on. When I run a puppet update, it completes without error. When I switch back to the puppet secondary-master, I get the duplicate class error.

They are both running puppet 3.8.7-1 on CentOS 6.
The YAML produced by both is exactly 100% the same. So I can assume the YAML structure is not the issue.

Would this suggest that the puppet secondary-master server is the issue, or the client connecting to it is perhaps not always getting what it wants from the slave?
Remember that the puppet updates will complete correctly for many hours, then magically change to this error. And vice-versa, be in error for many hours, and then magically change to completing correctly. Also that sometime changing configuration in The Forman can trigger the Error to occur AND other times trigger to Error to stop occurring.
Also note, I only have this problem with the saz-rsyslog module - NEVER with any other puppet module.
When I remove the saz-rsyslog module, all issues disappear.

I have made sure the puppet modules are 100% in sync between primary and secondary master server.
And I have restarted the puppet processes on the secondary-master server, but the error will continue on the nodes.

At this particular moment, all of the nodes experiencing the duplicate class declaration error are accessing the secondary-master puppet server. Not all nodes connecting to this secondary-master are experiencing this issue however - only 13 out of 33 nodes.
Also, I have other secondary-master servers on other segments. And nodes getting updates from those have experienced this error before too. However, I do not recall it occurring recently.
I cannot remember if a node connecting to the primary-master has experienced this issue - I want to say yes, but cannot recall to be certain.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

jcbollinger

unread,
Oct 10, 2016, 9:24:40 AM10/10/16
to Puppet Users


On Thursday, October 6, 2016 at 9:04:14 AM UTC-5, re-g...@wiu.edu wrote:
 
I understand the duplicate declaration issue - I have been reading about it for months trying to figure out this issue that keeps popping up. I am trying to figure out how it is possible I am getting a duplicate class declaration on a widely-used and approved module I have no control over (that no one else has reported any similar issue as mine), when using the forman to simply provide the values of parameters.



I don't know how widely-tested that module is, but it absolutely does exhibit what I would consider to be some violations of best practices.  As a result, it is more susceptible than most to duplicate declaration issues.  Nevertheless, it appears that simple uses of that module (e.g., of its classes, only ::rsyslog is declared directly) and all uses relying exclusively on automated data binding for parameter values should be ok.

 
Maybe The Foreman is the cause of the ::rsyslog class to be declared more than once? Might you suggest this possibility?


I did suggest that possibility, inasmuch as I suggested that it might be emitting the ENC analog of resource-like class declarations for it.  That in itself would not be enough to produce the problem, but it is a likely contributor.  I don't see any reason to think that it would emit multiple declarations itself, but it does not need to do, because there are several other classes in the module that explicitly or implicitly declare class ::rsyslog.  If any of those is also declared then there is a potential conflict.

 
But then I think, how would it? And why do I have this issue for hours on a server, and then at some magic moment the issue just disappears? Why with two of my servers sitting 1 IP away from each other, one will have this issue, and the other will not?


I cannot answer that in any detail, but obviously something differs over time and / or between servers.  There is an enormous variety of possibilities, both inside Puppet and / or The Foreman and outside.

 
I consider that The Foreman is causing the problem, but then I should have the error on all hosts at the same time.


I am by no means certain that the problem is directly related to The Foreman, and in no way am I suggesting that The Foreman is misbehaving.  I am, however, suggesting that in some cases The Foreman's ENC output is interacting poorly with the module.  That it is possible for that to be the case is a consequence of details of the module implementation.

 
The only different in the YAML between hosts is the IP addresses... every other puppet module parameter value is exactly the same.



That The Foreman is emitting a class declaration with parameters at all is very likely a contributing factor to the problem.  Note, however, that it may not be just the parameter values that make a difference.  It surely also matters which other classes are declared, and the order of the class declarations could also have an impact.  Again, however, I see no reason to attribute any of this to a flaw in The Foreman.

 
The Foreman has parameterized class support. It allows us to manage the values of the puppet module parameters.


Yes, and when you use that support, and it works exactly as intended, you open yourself up to duplicate class declaration issues.  Whether any such issues will actually manifest depends on many factors.

 
Old Screen Shots with explaination

When using the foreman, I am not writing any files, or declaring any classes. Instead, I manage the values of the puppet module parameters in the The Foreman WebUI, which is simply logic like 'if this host fact meets this criteria, then this puppet module parameter is set to this value'.


Even supposing that everything works exactly as designed, you don't need to write any of your own classes to end up with duplicate declaration issues.  You do need to feed class parameters to Puppet in some way (any way) other than automated data binding, and you need additional declarations of the same classes, whether implicit or explicit, in other classes you declare.  You certainly have the former, and the module you're using provides easy opportunities for the latter.


John

jcbollinger

unread,
Oct 10, 2016, 10:07:35 AM10/10/16
to Puppet Users


On Friday, October 7, 2016 at 10:39:58 AM UTC-5, re-g...@wiu.edu wrote:
Well... The node I have been testing the duplicate declaration on uses a puppet secondary-master server, as it is on a remote network segment. It does not connect directly to the puppet primary-master in which The Forman is running on.

So I did some work to get this particular "server1" node to use the puppet primary-master that The Foreman is running on. When I run a puppet update, it completes without error. When I switch back to the puppet secondary-master, I get the duplicate class error.

They are both running puppet 3.8.7-1 on CentOS 6.
The YAML produced by both is exactly 100% the same. So I can assume the YAML structure is not the issue.

Would this suggest that the puppet secondary-master server is the issue, or the client connecting to it is perhaps not always getting what it wants from the slave?
Remember that the puppet updates will complete correctly for many hours, then magically change to this error. And vice-versa, be in error for many hours, and then magically change to completing correctly. Also that sometime changing configuration in The Forman can trigger the Error to occur AND other times trigger to Error to stop occurring.
Also note, I only have this problem with the saz-rsyslog module - NEVER with any other puppet module.
When I remove the saz-rsyslog module, all issues disappear.


I am not prepared to believe that identical implementations of Puppet's catalog builder running on substantially identical platforms with identical inputs behave differently.  Since you do see variations in behavior, therefore, I conclude that those differences can be attributed to differences in implementation, platform, or (most likely) inputs.

 

I have made sure the puppet modules are 100% in sync between primary and secondary master server.
And I have restarted the puppet processes on the secondary-master server, but the error will continue on the nodes.



Those are good steps.  To really troubleshoot this thoroughly, however, I think you'll need to be systematic: capture the ENC output for each catalog request for a given node (or for all nodes), with timestamps and associated success / failure of catalog compilation.  Compare the ENC output for successful catalog builds with that for failed builds and look for systematics.

Either at the same time or separately, you should look into whether Puppet's environment cache has an impact here.  Some of the behaviors you describe make me rather suspicious of this.  Supposing that you are using directory environments, you should experiment both with disabling caching altogether (set the environment_timeout configuration option to 0 (its default)) and with caching indefinitely (set environment_timeout to "unlimited").  Note that Puppet recommends against using any other setting for that option.  You could also try turning on the ignorecache option at the master.


John

re-g...@wiu.edu

unread,
Oct 11, 2016, 4:54:25 PM10/11/16
to Puppet Users
So I ran a `puppet agent -t` on one of the problem nodes against the primary master puppet server (which was successful), and then afterwards the secondary master puppet server (which produces the duplicate declaration error for Class[Rsyslog]).

The size of both catalog files are exactly the same (I am referring to this file: /var/lib/puppet/client_data/catalog/server1.mydomain.example.com.json ).
The only difference inside the file is the order of items in the json arrays.

I have checked carefully, the order in which Rsyslog classes definitions are appearing in '.data.edges[]' and '.data.resources[]' is in the same order of appearance between catalogs from both servers, however they are intermingled with definitions from other modules differently - with this I mean the definition for Class[Rsyslog::Config] appears as element .data.edges[23].target in the primary master (that is successful), but as element .data.edges[4].target in the secondary master (that fails with duplicate declaration), though no other Rsyslog class comes before this one in each catalog.

So the only difference I can see between the two puppet servers is the order of the overall elements in the catalogs' json hashes and arrays.
Could this be a cause of the duplicate declaration error?
And if it might be, how do I get the definitions in the catalog to compile into the correct order that would prevent the duplicate declaration error?

jcbollinger

unread,
Oct 12, 2016, 8:59:25 AM10/12/16
to Puppet Users


On Tuesday, October 11, 2016 at 3:54:25 PM UTC-5, re-g...@wiu.edu wrote:


On Monday, October 10, 2016 at 9:07:35 AM UTC-5, jcbollinger wrote:


On Friday, October 7, 2016 at 10:39:58 AM UTC-5, re-g...@wiu.edu wrote:
Well... The node I have been testing the duplicate declaration on uses a puppet secondary-master server, as it is on a remote network segment. It does not connect directly to the puppet primary-master in which The Forman is running on.

So I did some work to get this particular "server1" node to use the puppet primary-master that The Foreman is running on. When I run a puppet update, it completes without error. When I switch back to the puppet secondary-master, I get the duplicate class error.

They are both running puppet 3.8.7-1 on CentOS 6.
The YAML produced by both is exactly 100% the same. So I can assume the YAML structure is not the issue.

Would this suggest that the puppet secondary-master server is the issue, or the client connecting to it is perhaps not always getting what it wants from the slave?
Remember that the puppet updates will complete correctly for many hours, then magically change to this error. And vice-versa, be in error for many hours, and then magically change to completing correctly. Also that sometime changing configuration in The Forman can trigger the Error to occur AND other times trigger to Error to stop occurring.
Also note, I only have this problem with the saz-rsyslog module - NEVER with any other puppet module.
When I remove the saz-rsyslog module, all issues disappear.


I am not prepared to believe that identical implementations of Puppet's catalog builder running on substantially identical platforms with identical inputs behave differently.  Since you do see variations in behavior, therefore, I conclude that those differences can be attributed to differences in implementation, platform, or (most likely) inputs.

 

I have made sure the puppet modules are 100% in sync between primary and secondary master server.
And I have restarted the puppet processes on the secondary-master server, but the error will continue on the nodes.



Those are good steps.  To really troubleshoot this thoroughly, however, I think you'll need to be systematic: capture the ENC output for each catalog request for a given node (or for all nodes), with timestamps and associated success / failure of catalog compilation.  Compare the ENC output for successful catalog builds with that for failed builds and look for systematics.

Either at the same time or separately, you should look into whether Puppet's environment cache has an impact here.  Some of the behaviors you describe make me rather suspicious of this.  Supposing that you are using directory environments, you should experiment both with disabling caching altogether (set the environment_timeout configuration option to 0 (its default)) and with caching indefinitely (set environment_timeout to "unlimited").  Note that Puppet recommends against using any other setting for that option.  You could also try turning on the ignorecache option at the master.


John


So I ran a `puppet agent -t` on one of the problem nodes against the primary master puppet server (which was successful), and then afterwards the secondary master puppet server (which produces the duplicate declaration error for Class[Rsyslog]).

The size of both catalog files are exactly the same (I am referring to this file: /var/lib/puppet/client_data/catalog/server1.mydomain.example.com.json ).
The only difference inside the file is the order of items in the json arrays.



A duplicate declaration error is a parse (i.e. catalog building) error.  If the master encounters one then it does not emit a catalog -- or if it did, it could not be one based on the failed catalog-building attempt.  I'm not sure what you're looking at, but it does not have the significance you are attributing to it.

 
So the only difference I can see between the two puppet servers is the order of the overall elements in the catalogs' json hashes and arrays.
Could this be a cause of the duplicate declaration error?


No.  Any catalog you have is the result of a catalog-building run in which no such error was produced.

The appearance of duplicate declaration errors can be sensitive to the order of declarations in your manifests or ENC output, but a catalog does not directly inform about that.


John

Henrik Lindberg

unread,
Oct 12, 2016, 11:10:55 AM10/12/16
to puppet...@googlegroups.com
Just a thought. If it first works and then stops working it could be
that the logic realizes exported resources and that it clashes with a
resource already created/realized.

- henrik


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

re-g...@wiu.edu

unread,
Oct 12, 2016, 5:15:51 PM10/12/16
to Puppet Users
You were right, that catalog file (server1:/var/lib/puppet/client_data/catalog/server1.mydomain.example.com.json) was from a successful build/run. There is no new catalog file when the server1 node receives the duplicate declaration error.

I did some further testing, and I hope I made a little progress on narrowing down the issue - perhaps this adds a clue:

Summary:
 1. Both rsyslog and rsyslog::client are assigned to the node server1 via a class group in The Foreman
 2. When I remove the rsyslog::client assignment, the server1 node can perform a successful puppet update, without a duplicate declaration error
 3. Then I added rsyslog::client assignment back into the group, I received the duplicate declaration error
 4. Then, one parameter at a time, I configured The Foreman to use the "default value" (checked the "use default" box) of each configured parameter for the node server1, when this fact matches: fqdn=server1.mydomain.example.com
 5. When all parameters are set to be "default value" for this node, even though the class is still assigned to the node via The Foreman, and even though the parameters are still set to a value for other nodes via The Foreman, the puppet update runs successfully without a duplicate declaration error on node server1
 6. After this, my going back and setting just one parameter in the rsyslog::client class for this node causes the duplicate declaration error

Details:

In The Foreman, I have a Puppet Class config group that I assign to the group of hosts in this remote location. I assign this same config group for every node in the network, even ones in the local network and other remote networks that do not experience duplicate class errors. Let me call this group DefG1.

The DefG1 config group has my default puppet module classes defined in it, including the Rsyslog puppet module class.
However, I am including a subclass rsyslog::client in addition to the main class rsyslog in this group DefG1. Because I need to configure the rsyslog client parameters for every node, namely the remote syslog server host.
Now, this is not an issue with other puppet modules to include sub modules (for example I also use both foreman-puppet and foreman-puppet::config together without issue). Nor has this been an issue for nodes performing puppet updates against the primary-master puppet server and all the other secondary-master puppet servers (I have 3 secondary-masters, 1 local network, and 3 remote networks with secondary-puppet servers).

I cannot configure the class parameters in The Forman in the puppet module's class for a node unless the puppet class is assigned to the node. I can assign the class (1) directly to the node, or (2) to the node group the node is in, or (3) to a class config group that I can assign to the node or the node group. I have done #3, the config group, and I assign the config group to the highest parent node group all nodes in all networks are a member of.


So when any rsyslog::client class parameters are defined, I receive the duplicate declaration error for nodes only in this one remote network. Yesterday I was experiencing this duplicate declaration error in another one of my remote networks, but it resolved itself magically within 24 hours - no one touched The Foreman, and all I did was login and monitor the dashboard. 

-RG

re-g...@wiu.edu

unread,
Oct 12, 2016, 5:22:18 PM10/12/16
to Puppet Users
So, I have 4 networks. one network has the primary-master puppet server, the other 3 have a secondary-master puppet server. Why do you think only one of my secondary-master puppet servers would have this issue most of the time, another secondary-master only some times, and a third secondary-master and the primary-master not at all.

This such that a puppet update on a node from its local secondary-master puppet server experienced the duplicate declaration. But running the puppet update immediately after that but from the primary-master works successfully without error.

Would you suppose I have puppet configured differently some how? And if so, what configuration can I look at to try and find the root cause of this issue?
Perhaps I can compare some configuration between the working secondary-master and the non-working secondary master? What config might this be?

All secondary-master puppet servers are identical, except of course for host name and IP address.

jcbollinger

unread,
Oct 13, 2016, 10:01:53 AM10/13/16
to Puppet Users


On Wednesday, October 12, 2016 at 4:15:51 PM UTC-5, re-g...@wiu.edu wrote:
 
You were right, that catalog file (server1:/var/lib/puppet/client_data/catalog/server1.mydomain.example.com.json) was from a successful build/run. There is no new catalog file when the server1 node receives the duplicate declaration error.

I did some further testing, and I hope I made a little progress on narrowing down the issue - perhaps this adds a clue:

Summary:
 1. Both rsyslog and rsyslog::client are assigned to the node server1 via a class group in The Foreman
 2. When I remove the rsyslog::client assignment, the server1 node can perform a successful puppet update, without a duplicate declaration error


That confirms what I have been saying all along.  Class rsyslog::client is a subclass of rsyslog (which is not true of most of the other rsyslog::* classes).  If you declare that class at all, with or without parameters, then you must not declare any parameters for class rsyslog.  If you want non-default parameters for class rsyslog then it must get them via automated data binding.

Technically, in fact, the module violates the DSL specifications by having rsyslog::client inherit from a parameterized class at all.  In practice, that combination normally works as desired, provided that any parameterized direct declaration of the base class is evaluated before all declarations of the subclass.  Evaluation-order dependencies such as this can be very difficult to deal with, and should be avoided at all costs.  This is the primary reason why the resource-like class declaration syntax, as it was designed and implemented, was a bad idea from the very beginning.

 
 3. Then I added rsyslog::client assignment back into the group, I received the duplicate declaration error
 4. Then, one parameter at a time, I configured The Foreman to use the "default value" (checked the "use default" box) of each configured parameter for the node server1, when this fact matches: fqdn=server1.mydomain.example.com
 5. When all parameters are set to be "default value" for this node, even though the class is still assigned to the node via The Foreman, and even though the parameters are still set to a value for other nodes via The Foreman, the puppet update runs successfully without a duplicate declaration error on node server1


I suspect that if you look at the output of Foreman's ENC then you will find that the declaration of class rsyslog::client is different in the case where all parameters are set to take "default value" than when any parameter is assigned an explicit value.  Although I have not probed this before, it would be logical for Puppet to evaluate the ENC-derived declarations in this order:
  1. global variable assignments
  2. class declarations with explicit parameters (which are in a different section of the ENC output from ...)
  3. class declarations without explicit parameters
If indeed Puppet does this, then that accounts for the behavior you describe by ensuring that class rsyslog is evaluated before rsyslog::client when no parameter values are specified for the latter.  (I re-emphasize that this makes a difference only because the module does not conform to the DSL specifications.) It seems that when both classes have parameter values assigned to them via The Foreman, then rsyslog::client is evaluated first, yielding the error; I have speculations about why that may be, but no solid information.

 
 6. After this, my going back and setting just one parameter in the rsyslog::client class for this node causes the duplicate declaration error



Yes, that's consistent with everything else you said.

 
Details:

In The Foreman, I have a Puppet Class config group that I assign to the group of hosts in this remote location. I assign this same config group for every node in the network, even ones in the local network and other remote networks that do not experience duplicate class errors. Let me call this group DefG1.

The DefG1 config group has my default puppet module classes defined in it, including the Rsyslog puppet module class.
However, I am including a subclass rsyslog::client in addition to the main class rsyslog in this group DefG1. Because I need to configure the rsyslog client parameters for every node, namely the remote syslog server host.
Now, this is not an issue with other puppet modules to include sub modules (for example I also use both foreman-puppet and foreman-puppet::config together without issue).


Again, the saz-rsyslog module has flaws that make its use susceptible to the kind of error you are experiencing.  The problem is not inherent in direct use, with or without parameters, of multiple classes from the same module, nor does it reflect a bug in Puppet or The Foreman.  One could wish that The Foreman would supply class parameters by hooking the automated data binding framework instead of by explicit declaration in its ENC output, but there would be both advantages and disadvantages to that from the Foreman perspective.

 
Nor has this been an issue for nodes performing puppet updates against the primary-master puppet server and all the other secondary-master puppet servers (I have 3 secondary-masters, 1 local network, and 3 remote networks with secondary-puppet servers).



I can only speculate about why you see the errors only on one of your secondary masters, with the primary and other secondaries never exhibiting it when building catalogs for the same nodes based on the same manifests, data, and Foreman configuration.  If that's indeed what you see, then the only possible explanation is that there is something different about the machine that exhibits the errors.  My first guess there would be that the version of Ruby on which the master and / or The Foreman is running differs.  My second would be that something about the way that saz-rsyslog configured into The Foreman differs (general configuration, not its use with any particular node).  I'm sure there are other possibilities.

 
I cannot configure the class parameters in The Forman in the puppet module's class for a node unless the puppet class is assigned to the node. I can assign the class (1) directly to the node, or (2) to the node group the node is in, or (3) to a class config group that I can assign to the node or the node group. I have done #3, the config group, and I assign the config group to the highest parent node group all nodes in all networks are a member of.


So when any rsyslog::client class parameters are defined, I receive the duplicate declaration error for nodes only in this one remote network. Yesterday I was experiencing this duplicate declaration error in another one of my remote networks, but it resolved itself magically within 24 hours - no one touched The Foreman, and all I did was login and monitor the dashboard. 



I cannot explain why the behavior would vary over time.  I have a few general ideas, but they are too speculative to discuss.

I think I've given you a pretty good handle on the nature of the problem, and I'm inclined to think that a solution that addresses the problem at its root will take care of the whole suite of issues.


John

Rob Nelson

unread,
Oct 13, 2016, 10:15:58 AM10/13/16
to puppet...@googlegroups.com
I've worked with saz in the past and believe he would be very receptive to PRs that address this issue, as well, if RG or anyone else wanted to contribute them. That would be one of the better solutions to the problem.

re-g...@wiu.edu

unread,
Oct 14, 2016, 6:08:31 PM10/14/16
to Puppet Users
Lots and lots of thanks to jcbollinger.
Here is what I have done as a result of this thread:

Submitted an issue with saz-rsyslog:
Duplicate Declaration Class[Rsyslog] with The Foreman as ENC during catalog compilation #237

Posted a follow-up question to a related thread on The Foreman mail list to determine if The Foreman also may be exhibiting a bug, or if the issue may be my configuration:
Duplicate declaration error.
https://groups.google.com/d/topic/foreman-users/k0JgABtszdU/discussion   (my post is waiting for approval as of 22:00 GMT 10/14/2016)

My current thought at this time is two points:
1. saz-rsyslog is written in such a way that declaring a subclass before the parent class causes duplicate declaration errors - but I am told the module inheritance should be addressed, and would resolve this issue.
2. The Foreman, under undetermined circumstances (maybe my misconfiguration), will randomly declare parent classes and subclass out of order - but I am told this is not an issue with puppet modules that use inheritance correctly - thus there might be an underlying bug with The Foreman because this error will not appear under modules' expected inheritance.

re-g...@wiu.edu

unread,
Oct 17, 2016, 4:15:51 PM10/17/16
to Puppet Users
This is what was posted on The Foreman mail list group:
This is an issue with the format of the ENC YAML used between Foreman 
and Puppet, and is best fixed in the module. 
The list of classes is given as a hash/dictionary and so has no 
particular order defined - it's down to the Puppet master/server to 
iterate over it, and it probably does so in no particular order. It 
isn't under Foreman's control.

I now believe this is truly the root of my problem. 

The evidence is looking at the ENC output and the Puppet-generated Catalog file on each of the master puppet servers.
As you can see below, the puppet-generated catalog on the secondary-master puppet server has put rsyslog::client before rsyslog - this causes the failure.
However, The Foreman ENC output is the same on both primary and secondary master puppet servers.

Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?
If this is correct, then this is the root of the problem.

The next question is, why does Puppet assemble a catalog with classes out of order, when the ENC output has them in the correct order?
I have seen two answers to this question of classes getting out of order in the puppet catalog:
 1. one class 'cA' depends on 'cB', so puppet puts 'cB' first in the catalog. - but this is not the reason in my case.
 2. "... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..." - which should be unlikely since it works on one secondary master and not the next secondary master, both configured exactly the same.

Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?


Running `puppet agent -t` on the client against the primary master puppet server is successful
Running `puppet agent -t` on the client against the secondary master puppet server fails with duplicate declaration


On the primary master puppet server

--The Foreman ENC output--
primary-puppet:/var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml
...snip...
    rsyslog: 
      gnutls_package_name: false
      preserve_fqdn: true
      relp_package_name: false
      rsyslog_package_name: false
    "rsyslog::client": 
      log_local: true
      remote_type: udp
...snip...
--end--

--Puppet generated catalog--
primary-puppet:/var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml
    rsyslog: 
      gnutls_package_name: false
      preserve_fqdn: true
      relp_package_name: false
      rsyslog_package_name: false
    "rsyslog::client": 
      log_local: true
      remote_type: udp
--end--


On the secondary master puppet server

--The Foreman ENC output--
secondary-puppet:/var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml
...snip...
  rsyslog:
    gnutls_package_name: false
    preserve_fqdn: true
    relp_package_name: false
    rsyslog_package_name: false
  rsyslog::client:
    log_local: true
    remote_type: udp
...snip...
--end--

--Puppet generated catalog--
secondary-puppet:/var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml
...snip...
    "rsyslog::client": 
      remote_type: udp
      log_local: true
    ntp: 
      server_list: 
    rsyslog: 
      gnutls_package_name: false
      rsyslog_package_name: false
      preserve_fqdn: true
      relp_package_name: false
...snip...
--end--


-RG

Rob Nelson

unread,
Oct 17, 2016, 6:00:14 PM10/17/16
to puppet...@googlegroups.com
RG,

I ultimately do not know the answer to your question (though I believe it may be due to the non-deterministic ordering when resources are not specifically ordered), however you should note that your primary service includes "rsyslog::client" and the secondary includes rsyslog::client (without the quotes). I am not sure if that is a red herring or if it means there is an actual difference in the delivered catalog and how puppet agent evaluates it, though it does seem odd that the ENC output has that slight variation between systems.


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/39fa6d99-eb37-4fb1-8e9c-7d04dcf0a574%40googlegroups.com.

jcbollinger

unread,
Oct 18, 2016, 10:49:23 AM10/18/16
to Puppet Users


You have taken a step backward there.  I remind you that duplicate declaration errors occur during catalog building, and that catalog building fails if one occurs.  It is therefore impossible to compare a catalog from the failure case to a catalog from the success case, because no catalog is ever created in the failure case.

It is useful to confirm that Foreman's ENC produces consistent output -- if indeed it does.  I'm not surprised that you obtained identical ENC output from two different Foreman servers, but it's unclear to me whether you have compared the failure-case ENC output with the success-case ENC output.  I am inclined to guess that these do differ, because clearly something changes.

Unless ....

Do check the the 'ordering' parameter in the [master] section of your master's puppet configuration.  If it is specified, and the specified value is anything other than 'manifest', then it is possible that setting its value to 'manifest' (the default) will help.  I emphasize, however, that this is a workaround for sloppy manifest code, not a bona fide solution.

You've made a lot of hay about saz-rsyslog being approved / known-good / high-quality, with no particular justification.  Certainly the module being available from the Forge should not be interpreted to support such a conclusion.  To be clear: having looked at the code of the module in question, I think it is generally of good quality.  As I have already said, however, and as Dominic Cleal told you over on the Foreman group, the fundamental problem here is in the manifests -- the module's manifests.  The module is designed and its manifests are written in a manner that is especially conducive to the kind of problem you've encountered.

 
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?
If this is correct, then this is the root of the problem.


I'm not sure what significance you're attributing to those particular files, but it's probably the wrong significance.  Yes, to the best of my knowledge and understanding, Puppet generates those files based on the node's certificate and facts, and the associated ENC output (if any).  But no, there is no reason to think that the root of your problem lies here.

I have already explained the root of the problem several times, but I'll try just once more: the saz-rsyslog module is designed and implemented in a way that does not support declaring class ::rsyslog via a resource-like declaration (whether Puppet DSL's version of that or its ENC analog) while also declaring any of several other public classes from the same module.  If there is any flaw to be attributed to Puppet here, it is that catalog building sometimes works for you anyway, not that it sometimes fails in the face this kind of misuse.

 

The next question is, why does Puppet assemble a catalog with classes out of order, when the ENC output has them in the correct order?


The order in which classes are listed in the ENC output is not significant, nor indeed is the order of entries in any YAML hash logically significant.  That's basically the answer.

Depending on several underlying factors, it might be that under some circumstances you do reliably see classes specified by the ENC being evaluated in an order consistent with the lexical order of the ENC output.  As I remarked in my previous message, the version of Ruby on which your master is running is one of my first guesses as to a source differences in that respect.

 
I have seen two answers to this question of classes getting out of order in the puppet catalog:


Classes are not getting out of order in the puppet catalog, nor indeed are the files that you earlier asked about (serialized) Puppet catalogs.

 
 1. one class 'cA' depends on 'cB', so puppet puts 'cB' first in the catalog. - but this is not the reason in my case.
 2. "... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..." - which should be unlikely since it works on one secondary master and not the next secondary master, both configured exactly the same.


You're now diverting to a different topic: why the observed behavior changes over time, as opposed to why it differs between masters.  I have no reason to doubt your assertion that the manifests involved are the same everywhere.  The data I refer to would be Puppet external data, which it seems you are not using, so that is trivially the same from run to run.  But that still leaves many things that could vary over time; the list you quoted covers other likely candidates, but it was never intended to be exhaustive.  If I had to guess, though, I'd place my bet on the ENC output varying.  Do note that it doesn't have to be the part of the ENC output that designates classes rsyslog and rsyslog::client that changes; a variety of other changes might result in an inversion of the order in which those two are evaluated, too.

 
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?


There is an unsupported-use-of-module issue ultimately causing all your problems, as I have been saying since the beginning.  It would be best to solve that problem, either by choosing a different module or by restricting yourself to supported uses of the module you have chosen (e.g. by using automated data binding to assign values to class parameters).

If you insist on continuing in your misuse and trying to work around the problems that follow from that, then it seems reasonable to focus on differences between the secondary master that fails and those that always work in this regard, if in fact they really do.  Compare software versions (especially of Puppet and of the Ruby on which Puppet is running (and watch out for multiple Ruby versions being co-installed).  Compare (puppet) config files.  Heck, replace the troublesome secondary master with a clone of one of the others if you can't figure out what differs.

 
Running `puppet agent -t` on the client against the primary master puppet server is successful
Running `puppet agent -t` on the client against the secondary master puppet server fails with duplicate declaration

 

I note in passing that you've now shifted back to the topic of behavior differing between different masters.



  1. Those are not catalogs you are looking at.
  2. You've presented excerpts of the ENC output and node dumps as if those are the only parts that matter.  The totality of the ENC output matters (but I don't think the node dumps actually matter at all).

John

re-g...@wiu.edu

unread,
Oct 19, 2016, 2:05:54 PM10/19/16
to Puppet Users
On Monday, October 17, 2016 at 5:00:14 PM UTC-5, Rob Nelson wrote:
RG,

I ultimately do not know the answer to your question (though I believe it may be due to the non-deterministic ordering when resources are not specifically ordered), however you should note that your primary service includes "rsyslog::client" and the secondary includes rsyslog::client (without the quotes). I am not sure if that is a red herring or if it means there is an actual difference in the delivered catalog and how puppet agent evaluates it, though it does seem odd that the ENC output has that slight variation between systems.


Argh! Sorry about that, that was my copy-paste error.
I went back and double-checked. The primary-puppet foreman output with without the quotes. The primary-puppet node output is with the quotes. in the node output, every subclass is enclosed in double-quotes because it has double-colons in the name string.
The primary and secondary puppet masters have the same foreman output file, which is the same for rsyslog,rsyslog::* in the primary-puppet node output file. I must have copied from the wrong screen because they looked the same, and then overlooked the double-quotes when double-checking my work.


To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

re-g...@wiu.edu

unread,
Oct 19, 2016, 5:51:20 PM10/19/16
to Puppet Users
Then I was mistaken in thinking that the node file (/var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml) was the data (catalog pre-data) written to a file before the catalog compilation procedure.
I assumed this because both the foreman and node files are created new every time server1 performs a puppet agent update, even when the result is a 400 error for the catalog compilation.
If I am mistaken in this, then I need someone to tell me how to get the data of the catalog that is failing to be compiled, so that I can compare it with other sources.

Here is the procedure I have been assuming (I am not an expert on puppet internals):
1. (server1)puppet-agent -> puppet-master
2. puppet-master -> (exec)puppet[external_nodes] server1.mydomain.example.com -> foreman ENC
3. foreman ENC -> /var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml -> puppet-master
4. puppet-master -> /var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml -> puppet-master(compile-catalog)
5. puppet-master(compile-catalog) -> Error -> 400 Server Error -> (server1)puppet-agent

I was assuming the node file is the data of the catalog to be compiled. I had noticed it is updated in every puppet agent run, even when the run is in error. And it was the differing piece between primary and secondary puppet masters.
Please correct me.
Please let me know how to get the data of the catalog the puppet master is failing to compile for the puppet agent.
 
It is useful to confirm that Foreman's ENC produces consistent output -- if indeed it does.  I'm not surprised that you obtained identical ENC output from two different Foreman servers, but it's unclear to me whether you have compared the failure-case ENC output with the success-case ENC output.  I am inclined to guess that these do differ, because clearly something changes.


The /var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml file is the same on both
The (exec)puppet[external_nodes] server1.mydomain.example.com (/etc/puppet/node.rb server1.mydomain.example.com) output is the same on both - I assume this is the only source and final data from The Foreman ENC.
 
Unless ....

Do check the the 'ordering' parameter in the [master] section of your master's puppet configuration.  If it is specified, and the specified value is anything other than 'manifest', then it is possible that setting its value to 'manifest' (the default) will help.  I emphasize, however, that this is a workaround for sloppy manifest code, not a bona fide solution.


No, this parameter is not set on either the primary or secondary puppet masters.
The only differing parameters are 'ca' and 'certname'.
Between a this secondary-master and another secondary-master (not having issues), only the `certname` parameter is different.

 
You've made a lot of hay about saz-rsyslog being approved / known-good / high-quality, with no particular justification.  Certainly the module being available from the Forge should not be interpreted to support such a conclusion.  To be clear: having looked at the code of the module in question, I think it is generally of good quality.  As I have already said, however, and as Dominic Cleal told you over on the Foreman group, the fundamental problem here is in the manifests -- the module's manifests.  The module is designed and its manifests are written in a manner that is especially conducive to the kind of problem you've encountered.


The saz-rsyslog module is officially "approved", because it is labeled as "approved" on the forge site. I used that term as an identifying piece of information. I myself do not give it that label.
If you search for saz-rsyslog on forge, you will see it as the first result with a 45 deg banner in the top right that reads "APPROVED".
I do not remember mentioning it is known-good or high-quality. I apologize if I gave that perception.
However, reading this source: https://forge.puppet.com/approved, the "APPROVED" label from Puppet indicates the module does "Adhere to Puppet's standards for style and design".

I do remember saying it was either widely used or popular. But I said this only as another "labeling" or identifying factor, as the same search results entry say it has been downloaded 36,000+ times.
Version 4.0.3 - Oct 4, 2016 - 36,632 downloads
Again, I apologize for unintended misconceptions I have given.
 
 
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?
If this is correct, then this is the root of the problem.


I'm not sure what significance you're attributing to those particular files, but it's probably the wrong significance.  Yes, to the best of my knowledge and understanding, Puppet generates those files based on the node's certificate and facts, and the associated ENC output (if any).  But no, there is no reason to think that the root of your problem lies here.


As I said earlier, I now assume I misunderstood this as being the file holding the data to be compiled as the catalog.
I apologize for the misunderstanding, but that I intended it as the "root" of the catalog-compilation "problem". You are right that it is not the root of the entire underlying problem.
Again, I need help to find the catalog data trying to be compiled, but which fails, if this /.../node/.. file is not that data.

However, the failure point is the catalog compilation. Nothing fails leading up to this, so it has been very hard to track down the causes leading up to this revealed error.
You are right, the solvable problem is fixing the inheritance issue of the puppet module.
 
I have already explained the root of the problem several times, but I'll try just once more: the saz-rsyslog module is designed and implemented in a way that does not support declaring class ::rsyslog via a resource-like declaration (whether Puppet DSL's version of that or its ENC analog) while also declaring any of several other public classes from the same module.  If there is any flaw to be attributed to Puppet here, it is that catalog building sometimes works for you anyway, not that it sometimes fails in the face this kind of misuse.


Yes, I agreed with that. I apologize for not confirming this conclusion.
Instead I responded by pointing to the bug I filed with saz-rsyslog. And in that saz-rsyslog bug report I provided your analysis of the issues, and pointed back at this thread.
Thank you for that. That was what I needed to be able to file a bug.
 
 

The next question is, why does Puppet assemble a catalog with classes out of order, when the ENC output has them in the correct order?


The order in which classes are listed in the ENC output is not significant, nor indeed is the order of entries in any YAML hash logically significant.  That's basically the answer.

Depending on several underlying factors, it might be that under some circumstances you do reliably see classes specified by the ENC being evaluated in an order consistent with the lexical order of the ENC output.  As I remarked in my previous message, the version of Ruby on which your master is running is one of my first guesses as to a source differences in that respect.


I checked the versions of ruby installed. They are both running:
Package: ruby193-1.1-9.sc1.el6.x86_64
Version: ruby 1.9.3p484 (2013-11-22) [x86_64-linux]
To top it off, we are mirroring the SCLo repositories with Pulp, and so both puppet master servers are pulling packages from the same static repository.
 
 
I have seen two answers to this question of classes getting out of order in the puppet catalog:


Classes are not getting out of order in the puppet catalog, nor indeed are the files that you earlier asked about (serialized) Puppet catalogs.


As I eluded to earlier, before I was assuming the node file (/var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml) was the the data that was to be compiled as the catalog. I need help to know how to get the data that is compiled for the catalog. If I can compare this data on both servers, perhaps that is a lead. I am grasping at straws.
  
 
 1. one class 'cA' depends on 'cB', so puppet puts 'cB' first in the catalog. - but this is not the reason in my case.
 2. "... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..." - which should be unlikely since it works on one secondary master and not the next secondary master, both configured exactly the same.


You're now diverting to a different topic: why the observed behavior changes over time, as opposed to why it differs between masters.  I have no reason to doubt your assertion that the manifests involved are the same everywhere.  The data I refer to would be Puppet external data, which it seems you are not using, so that is trivially the same from run to run.  But that still leaves many things that could vary over time; the list you quoted covers other likely candidates, but it was never intended to be exhaustive.  If I had to guess, though, I'd place my bet on the ENC output varying.  Do note that it doesn't have to be the part of the ENC output that designates classes rsyslog and rsyslog::client that changes; a variety of other changes might result in an inversion of the order in which those two are evaluated, too.


Yes, I was considering this before.
However, in my previous post, and as mentioned earlier in this reply, I had come to assume the foreman file (/var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml) was the ENC output that Puppet used in the catalog. And these files are the same on both the primary and secondary master servers.

And as mentioned earlier in this reply, I also checked the output of `(exec)puppet[external_nodes] server1.mydomain.example.com` (/etc/puppet/node.rb server1.mydomain.example.com) and they also match between primary and secondary puppet master servers. And that output put in a file matches exactly the content of the foreman file and the file size (/var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml).
 
 
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?


There is an unsupported-use-of-module issue ultimately causing all your problems, as I have been saying since the beginning.  It would be best to solve that problem, either by choosing a different module or by restricting yourself to supported uses of the module you have chosen (e.g. by using automated data binding to assign values to class parameters).


"restricting yourself to supported uses of the module"
So using The Foreman as ENC has not been an official tested use of the module and thus is not supported? This is true for all modules on forge I have reviewed, except for the ones published by The Foreman group.
To be supported, should I instead restrict my use of either declaring my classes in a site.pp or use hiera?

"using automated data binding to assign values to class parameters"
What is automated data binding? Is that what I get when using hiera? Would that mean I would forgo the use of The Forman to instead use hiera?
I have been assuming hiera was relatively the same as using The Foreman as ENC? Reference: https://ask.puppet.com/question/527/are-hiera-and-foreman-parameters-mutually-exclusive/
 
If you insist on continuing in your misuse and trying to work around the problems that follow from that, then it seems reasonable to focus on differences between the secondary master that fails and those that always work in this regard, if in fact they really do.  Compare software versions (especially of Puppet and of the Ruby on which Puppet is running (and watch out for multiple Ruby versions being co-installed).  Compare (puppet) config files.  Heck, replace the troublesome secondary master with a clone of one of the others if you can't figure out what differs.

 
Running `puppet agent -t` on the client against the primary master puppet server is successful
Running `puppet agent -t` on the client against the secondary master puppet server fails with duplicate declaration

 

I note in passing that you've now shifted back to the topic of behavior differing between different masters.


I am grasping at straws. I have been assuming what I can find different between the servers would reveal the issue.
I do not deny that the saz-rsyslog puppet module issues should be addressed, and would resolve the issues at hand.
But it seems that if I have the problem on one server, but not another server, there is something more that is wrong that goes beyond only the module's issues.
Thus I point at two possible issues. One with the module. Another with either The Foreman or Puppet.

And on top of that, the secondary-master puppet server is not consistent in always throwing the duplicate declaration error.

I guess I need someone to tell me that if a puppet module is having the issue we describe in this thread, that due to the intended and expected nature of some function/process "X" in The Foreman or Puppet, I can expect to see the module's inheritance issue cause catalog-compilation errors at random times.

Example: it was suggested that the hash keys are not always ordered the same. So due to python sometimes ordering the hash keys differently, a module with inheritance issues like saz-rsyslog will sometimes cause a catalog compilation error, and sometimes not be an issue. This means Puppet expects modules to not have an inheritance issue like we are describing in order to guarantee there will not be issues during deployment, like in my case.
So you are saying the node dump is not the catalog.
Please tell me how to obtain the catalog data that it is failing to compile due to duplicate declaration.

You have given me a lot of information. I greatly appreciate your help.
-RG
 

jcbollinger

unread,
Oct 20, 2016, 11:31:59 AM10/20/16
to Puppet Users


But you said you were looking at the catalog not "catalog pre-data". I'm sure you appreciate the tremendous difference between those.

 
I assumed this because both the foreman and node files are created new every time server1 performs a puppet agent update, even when the result is a 400 error for the catalog compilation.
If I am mistaken in this, then I need someone to tell me how to get the data of the catalog that is failing to be compiled, so that I can compare it with other sources.

Here is the procedure I have been assuming (I am not an expert on puppet internals):
1. (server1)puppet-agent -> puppet-master
2. puppet-master -> (exec)puppet[external_nodes] server1.mydomain.example.com -> foreman ENC
3. foreman ENC -> /var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml -> puppet-master
4. puppet-master -> /var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml -> puppet-master(compile-catalog)
5. puppet-master(compile-catalog) -> Error -> 400 Server Error -> (server1)puppet-agent

I was assuming the node file is the data of the catalog to be compiled. I had noticed it is updated in every puppet agent run, even when the run is in error. And it was the differing piece between primary and secondary puppet masters.
Please correct me.


The YAML files in question are just byproducts of catalog building, not, as I understand it, intermediate files in the catalog-building process.  In any event, an ENC certainly delivers its results to Puppet via its standard output, not by writing them to a regular file.  Nor would I expect any ENC to create or update files under /var/lib/puppet in any case.

 
Please let me know how to get the data of the catalog the puppet master is failing to compile for the puppet agent.


If you are not assigning any classes or resources directly via a site manifest, and you are not using Hiera or any other form of external data besides variables assigned via Foreman, then the facts reported by the node, the ENC output for that node, and the master's standard top-scope variables together constitute the data from which catalog building proceeds.  It is possible, however, for evaluation of specific manifests to result in arbitrary data being gleaned from the catalog-building environment by use of Puppet functions, such as generate(), which can run an external command, and template() / inline_template(), which can run arbitrary Ruby code.

You can get the latest facts reported by a given node via the 'puppet facts find <node_certname>' command on the master.  If you're using puppetdb, then you can get them from there, too.

 
 
 
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?
If this is correct, then this is the root of the problem.


I'm not sure what significance you're attributing to those particular files, but it's probably the wrong significance.  Yes, to the best of my knowledge and understanding, Puppet generates those files based on the node's certificate and facts, and the associated ENC output (if any).  But no, there is no reason to think that the root of your problem lies here.


As I said earlier, I now assume I misunderstood this as being the file holding the data to be compiled as the catalog.
I apologize for the misunderstanding, but that I intended it as the "root" of the catalog-compilation "problem". You are right that it is not the root of the entire underlying problem.
Again, I need help to find the catalog data trying to be compiled, but which fails, if this /.../node/.. file is not that data.


The starting point for classes to be applied is the combination of ENC output with the site manifest.  I cannot speak to what may be in your site manifest.  What file or files constitute your site manifest depends on your Puppet configuration.  Some of the more likely possibilities are '<puppet root>/manifests/site.pp', '<puppet root>/environments/production/manifests/site.pp', or jointly all files of the form '<puppet root>/environments/production/manifests/*.pp'.  Within the site manifest, declarations at top scope and within the node block best matching the target machine (if any match at all) are relevant.

 
 
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?


There is an unsupported-use-of-module issue ultimately causing all your problems, as I have been saying since the beginning.  It would be best to solve that problem, either by choosing a different module or by restricting yourself to supported uses of the module you have chosen (e.g. by using automated data binding to assign values to class parameters).


"restricting yourself to supported uses of the module"
So using The Foreman as ENC has not been an official tested use of the module and thus is not supported? This is true for all modules on forge I have reviewed, except for the ones published by The Foreman group.
To be supported, should I instead restrict my use of either declaring my classes in a site.pp or use hiera?


Puppet supports declaring classes via an ENC generally.  Inasmuch as its implementation violates the DSL's documented constraints, however, the 'rsyslog::client' class is not supported at all in the sense of the term I was using.

Since avoiding that class is not a viable solution (unless by choosing an altogether different module), the next best thing would be to stick to uses that are well known to work despite not actually being supported -- in this case, that would be to avoid using (the ENC analog of) a resource-like declaration for the base class, rsyslog.  You have exactly one option for customizing that class's parameters without modifying it or using a resource-like declaration: automated data binding.

 

"using automated data binding to assign values to class parameters"
What is automated data binding? Is that what I get when using hiera?


It is documented here: https://docs.puppet.com/hiera/3.2/puppet.html#automatic-parameter-lookup.  It is based on Hiera, but it would not be quite correct to say it's "what I get when using hiera", because Hiera supports other uses as well.

 
Would that mean I would forgo the use of The Forman to instead use hiera?


It would mean that you forgo using The Foreman for setting the parameters of class rsyslog, at least for those nodes to which you also assign another class from the same module, such as rsyslog::client.  You should otherwise be able to continue using The Foreman as you currently do.

 
I have been assuming hiera was relatively the same as using The Foreman as ENC? Reference: https://ask.puppet.com/question/527/are-hiera-and-foreman-parameters-mutually-exclusive/
 


No, they are not the same thing at all, though they can serve many of the same purposes (and the answer you linked says pretty much the same thing).  Hiera is not an ENC, and Puppet does not use it in the same way or via the same interface that it uses an ENC.  Binding values to class parameters via (hiera-supported) automatic data binding has different semantics than does binding parameter values via an ENC.  From a Foreman perspective, you might view automated data binding as affecting the default values for class parameters.

 
I am grasping at straws. I have been assuming what I can find different between the servers would reveal the issue.


Indeed, the problems are mysterious, and it is entirely reasonable to suppose that differing behavior arises from discoverable differences between machines.  Myself, I had high hopes for an explanation in terms of differing versions of Ruby.  I guess there's still a chance that a difference in the installed Ruby modules might factor in.  And I wasn't kidding when I suggested replacing the machine on which the misbehavior is observed with a clone of one of those that behave as you want.

However, I also hold out some suspicion that the problem may not be in the machine at all, but rather in the inputs (ENC output varying over time; differences in other parts of the ENC output; node facts).  Or the difference may be in the dynamic operating environment, including Puppet runtime state.  I'm not sure how to test that directly, but it might be interesting to swap masters to see whether the problem sticks with the machine or with the workload.

 
I guess I need someone to tell me that if a puppet module is having the issue we describe in this thread, that due to the intended and expected nature of some function/process "X" in The Foreman or Puppet, I can expect to see the module's inheritance issue cause catalog-compilation errors at random times.

 
Example: it was suggested that the hash keys are not always ordered the same. So due to python sometimes ordering the hash keys differently, a module with inheritance issues like saz-rsyslog will sometimes cause a catalog compilation error, and sometimes not be an issue. This means Puppet expects modules to not have an inheritance issue like we are describing in order to guarantee there will not be issues during deployment, like in my case.


Well, the problem with hashes is that hash iteration order is not necessarily predictable in advance.  However, given the same hash implementation, initialized the same way, and subjected to the same sequence of additions and removals, you should see the same iteration order on every trial.  On the other hand, if the sequence of additions and removals changes, and especially if more, fewer, or different keys are added in different trials, then you cannot rely on iteration order to be consistent, even in part.  That's one reason why I earlier pointed out that the problem does not have to be in the portion of the ENC output that pertains directly to the rsyslog classes.

So no, I cannot tell you that with everything remaining the same, you should expect that Puppet's catalog building behavior might vary from run to run.  Quite the opposite.  But on the other hand, you can never have everything exactly the same.

I sympathize with you in your difficulty discovering the specific difference(s) that explain the variation in outcome, and some of my suggestions have been aimed at helping you there.  But you should not assume that figuring it out will lead you to a solution you like better than those I have proposed.  It might do.  It also might not.  And I cannot evaluate for you whether it's worthwhile to keep looking.


John

re-g...@wiu.edu

unread,
Oct 20, 2016, 5:42:47 PM10/20/16
to Puppet Users
Between primary and secondary masters, this information is the same, in the same order, and formatted the same (including spaces).
1. the facts reported by the node
2. ENC output for that node
3. master's standard top-scope variables

I only have the following set in the top scope, on all masters:
File: /etc/puppet/environments/production/manifests/site.pp
Package { allow_virtual => false }

puppet 'environments' directory is synced from primary to all secondary puppet master servers.
 

 
 
 
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?
If this is correct, then this is the root of the problem.


I'm not sure what significance you're attributing to those particular files, but it's probably the wrong significance.  Yes, to the best of my knowledge and understanding, Puppet generates those files based on the node's certificate and facts, and the associated ENC output (if any).  But no, there is no reason to think that the root of your problem lies here.


As I said earlier, I now assume I misunderstood this as being the file holding the data to be compiled as the catalog.
I apologize for the misunderstanding, but that I intended it as the "root" of the catalog-compilation "problem". You are right that it is not the root of the entire underlying problem.
Again, I need help to find the catalog data trying to be compiled, but which fails, if this /.../node/.. file is not that data.


The starting point for classes to be applied is the combination of ENC output with the site manifest.  I cannot speak to what may be in your site manifest.  What file or files constitute your site manifest depends on your Puppet configuration.  Some of the more likely possibilities are '<puppet root>/manifests/site.pp', '<puppet root>/environments/production/manifests/site.pp', or jointly all files of the form '<puppet root>/environments/production/manifests/*.pp'.  Within the site manifest, declarations at top scope and within the node block best matching the target machine (if any match at all) are relevant.


/etc/puppet/manifests/site.pp - empty
/etc/puppet/environments/production/manifests/site.pp - content provided earlier, and is same on all secondary puppet masters
There are no other .../manifests/*.pp files

 
 
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?


There is an unsupported-use-of-module issue ultimately causing all your problems, as I have been saying since the beginning.  It would be best to solve that problem, either by choosing a different module or by restricting yourself to supported uses of the module you have chosen (e.g. by using automated data binding to assign values to class parameters).


"restricting yourself to supported uses of the module"
So using The Foreman as ENC has not been an official tested use of the module and thus is not supported? This is true for all modules on forge I have reviewed, except for the ones published by The Foreman group.
To be supported, should I instead restrict my use of either declaring my classes in a site.pp or use hiera?


Puppet supports declaring classes via an ENC generally.  Inasmuch as its implementation violates the DSL's documented constraints, however, the 'rsyslog::client' class is not supported at all in the sense of the term I was using.

Since avoiding that class is not a viable solution (unless by choosing an altogether different module), the next best thing would be to stick to uses that are well known to work despite not actually being supported -- in this case, that would be to avoid using (the ENC analog of) a resource-like declaration for the base class, rsyslog.  You have exactly one option for customizing that class's parameters without modifying it or using a resource-like declaration: automated data binding.

 

"using automated data binding to assign values to class parameters"
What is automated data binding? Is that what I get when using hiera?


It is documented here: https://docs.puppet.com/hiera/3.2/puppet.html#automatic-parameter-lookup.  It is based on Hiera, but it would not be quite correct to say it's "what I get when using hiera", because Hiera supports other uses as well.

 
Would that mean I would forgo the use of The Forman to instead use hiera?


It would mean that you forgo using The Foreman for setting the parameters of class rsyslog, at least for those nodes to which you also assign another class from the same module, such as rsyslog::client.  You should otherwise be able to continue using The Foreman as you currently do.


Okay, I understand your recommendation.
To continue using this rsyslog module, I should use another method (that is supported, like Hiera) other than The Foreman to configure this class.

Unfortunately this is not ideal, since that makes two places to maintain configuration.
It would be better to find a replacement for the saz-rsyslog module, which you have previously recommended.

 
 
I have been assuming hiera was relatively the same as using The Foreman as ENC? Reference: https://ask.puppet.com/question/527/are-hiera-and-foreman-parameters-mutually-exclusive/
 


No, they are not the same thing at all, though they can serve many of the same purposes (and the answer you linked says pretty much the same thing).  Hiera is not an ENC, and Puppet does not use it in the same way or via the same interface that it uses an ENC.  Binding values to class parameters via (hiera-supported) automatic data binding has different semantics than does binding parameter values via an ENC.  From a Foreman perspective, you might view automated data binding as affecting the default values for class parameters.


Got it. That is clear.
 
 
I am grasping at straws. I have been assuming what I can find different between the servers would reveal the issue.


Indeed, the problems are mysterious, and it is entirely reasonable to suppose that differing behavior arises from discoverable differences between machines.  Myself, I had high hopes for an explanation in terms of differing versions of Ruby.  I guess there's still a chance that a difference in the installed Ruby modules might factor in.  And I wasn't kidding when I suggested replacing the machine on which the misbehavior is observed with a clone of one of those that behave as you want.

I combed through the servers. They are exactly the same down to the installed RPM packages. They were auto installed with The Foreman, via a kickstart. So their origins are the same.
However, I installed and configured The Foreman and secondary Puppet masters manually.
All Puppet configuration is maintained by puppet itself.


However, I also hold out some suspicion that the problem may not be in the machine at all, but rather in the inputs (ENC output varying over time; differences in other parts of the ENC output; node facts).  Or the difference may be in the dynamic operating environment, including Puppet runtime state.  I'm not sure how to test that directly, but it might be interesting to swap masters to see whether the problem sticks with the machine or with the workload.


I had pointed a client at the primary master, and it runs successfully. Only on the secondary puppet masters I have issues.

 
I guess I need someone to tell me that if a puppet module is having the issue we describe in this thread, that due to the intended and expected nature of some function/process "X" in The Foreman or Puppet, I can expect to see the module's inheritance issue cause catalog-compilation errors at random times.

 
Example: it was suggested that the hash keys are not always ordered the same. So due to python sometimes ordering the hash keys differently, a module with inheritance issues like saz-rsyslog will sometimes cause a catalog compilation error, and sometimes not be an issue. This means Puppet expects modules to not have an inheritance issue like we are describing in order to guarantee there will not be issues during deployment, like in my case.


Well, the problem with hashes is that hash iteration order is not necessarily predictable in advance.  However, given the same hash implementation, initialized the same way, and subjected to the same sequence of additions and removals, you should see the same iteration order on every trial.  On the other hand, if the sequence of additions and removals changes, and especially if more, fewer, or different keys are added in different trials, then you cannot rely on iteration order to be consistent, even in part.  That's one reason why I earlier pointed out that the problem does not have to be in the portion of the ENC output that pertains directly to the rsyslog classes.

So no, I cannot tell you that with everything remaining the same, you should expect that Puppet's catalog building behavior might vary from run to run.  Quite the opposite.  But on the other hand, you can never have everything exactly the same.

I sympathize with you in your difficulty discovering the specific difference(s) that explain the variation in outcome, and some of my suggestions have been aimed at helping you there.  But you should not assume that figuring it out will lead you to a solution you like better than those I have proposed.  It might do.  It also might not.  And I cannot evaluate for you whether it's worthwhile to keep looking.


John


After taking a look at the rsyslog puppet module code, it would seem saz expects either the rsyslog::client or rsyslog::server to be declared alone. Then inherit the parent rsyslog class so that "top-level" parameters (I use that generically) can be overwritten in your node scode.
Unfortunately, The Foreman see every class and subclasses independently, and lets you configure the parameters of each.

So, when The Foreman ENC sends a declaration for Rsyslog to Puppet, it is already wrong according to the module's expected use. But (like you were saying) the compilation will not necessarily fail if it so happens that Rsyslog is declared before Rsyslog::client in the puppet agent catalog - which is not necessarily dependent upon the ENC output which may have them declared in the correct order, but dependent upon puppet master's discovery of class declaration in the internal ruby hash created from ENC output (e.g. the order of hash keys) and the subsequent ordering of those classes during catalog compilation.

As best as I am able to figure out, the primary and secondary puppet servers receive the same ENC YAML data. So, somewhere after that YAML is hashed inside puppet master, this issue described with the saz-rsyslog module will cause the catalog compilation error should the Rsyslog::client class be declared before Rsyslog.


Being it might seem saz-rsyslog will stay as it is, it would seem my best course of action is to use an alternate rsyslog module if I wish to continue using The Foreman.


Thank you for all of your help.

-RG

re-g...@wiu.edu

unread,
Oct 21, 2016, 10:48:02 AM10/21/16
to Puppet Users
As it seems saz-rsyslog is the best puppet module for managing rsyslog, I decided to take a hint from @jcbollinger on configuration, rather than replacing the module.

I decided to change the values of the parent Rsyslog class parameters so that they are the seen as the "default" for The Foreman (I refer to "default" as eluded to in a previous reply regarding use of Hiera to change these values). But I did not want to implement Hiera just for one class and four parameters. So I added the following changes on all puppet master servers to the production/manifests/site.pp file:

File: /etc/puppet/environments/production/manifests/site.pp
class { 'rsyslog':
    preserve_fqdn        => true,
    gnutls_package_name  => false,
    relp_package_name    => false,
    rsyslog_package_name => false
}

This changes the values of these parameters globally, which is how I had it set in The Foreman any way.
And in The Foreman, I removed all values set for the parent Rsyslog class parameters, leaving them checked to use the default value. I left the values set in The Foreman for Rsyslog::client subclass parameters unchanged (which is necessary because these values change based on node facts). I did leave the rsyslog parent class assigned to all nodes in The Foreman, though the ENC output does not set any parameters for this class.

This fixed the duplicate declaration issue I was experiencing.
The reason why I believe this fixed the duplicate declaration error is because I assume Puppet's intended behavior is to always declare in the catalog the classes from the site.pp file before the classes from the ENC.

This is obviously not ideal, but I justify this with the argument that I am simply changing the "default" values of the saz-rsyslog puppet module Rsyslog class - but without having to modify the module's code.
This is a good enough work-around for me for now.
-RG

jcbollinger

unread,
Oct 24, 2016, 10:49:16 AM10/24/16
to Puppet Users


On Friday, October 21, 2016 at 9:48:02 AM UTC-5, re-g...@wiu.edu wrote:
 
As it seems saz-rsyslog is the best puppet module for managing rsyslog, I decided to take a hint from @jcbollinger on configuration, rather than replacing the module.

I decided to change the values of the parent Rsyslog class parameters so that they are the seen as the "default" for The Foreman (I refer to "default" as eluded to in a previous reply regarding use of Hiera to change these values). But I did not want to implement Hiera just for one class and four parameters. So I added the following changes on all puppet master servers to the production/manifests/site.pp file:

File: /etc/puppet/environments/production/manifests/site.pp
class { 'rsyslog':
    preserve_fqdn        => true,
    gnutls_package_name  => false,
    relp_package_name    => false,
    rsyslog_package_name => false
}

This changes the values of these parameters globally, which is how I had it set in The Foreman any way.
And in The Foreman, I removed all values set for the parent Rsyslog class parameters, leaving them checked to use the default value. I left the values set in The Foreman for Rsyslog::client subclass parameters unchanged (which is necessary because these values change based on node facts). I did leave the rsyslog parent class assigned to all nodes in The Foreman, though the ENC output does not set any parameters for this class.

This fixed the duplicate declaration issue I was experiencing.
The reason why I believe this fixed the duplicate declaration error is because I assume Puppet's intended behavior is to always declare in the catalog the classes from the site.pp file before the classes from the ENC.

This is obviously not ideal, but I justify this with the argument that I am simply changing the "default" values of the saz-rsyslog puppet module Rsyslog class - but without having to modify the module's code.
This is a good enough work-around for me for now.
-RG



I'm glad you have found a workaround that solves the problem for you.  I suspect that you are basically correct about why it works; that is, that Puppet evaluates the top-scope declaration of class ::rsyslog before it evaluates any class declarations from the ENC.

To make future trouble in this area less likely, however, I recommend that you adjust your mental model.  Unlike my recommended solution based on automated data binding, what you have done is not analogous to changing Foreman's view of the default values for the class's parameters.
  • In the first place, your class declaration in site.pp is indeed a bona fide class declaration, applying to every node.  As long as it's there, every node will get class `rsyslog`, with the parameters you've specified, regardless of what you specify via The Foreman.
  • In the second place, you now will reliably get duplicate declaration errors if you try to declare any non-default parameters for class rsyslog via Foreman.  That may actually be an improvement over getting such errors unpredictably, however.
Note also that your solution does not resolve the underlying problem of your set of declarations being sensitive to evaluation order.  Instead, it asserts sufficient control over the evaluation order to avoid the errors you experienced.  I find no documentation of the relative evaluation order you observe, and if it is undocumented then it is fair game for a change in some future version of Puppet.  (And indeed, Puppet has historically been pretty open to changing even documented behavior, albeit in a manner consistent with its commitment to semantic versioning.


John

Reply all
Reply to author
Forward
0 new messages