--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1ead58c4-ed05-4b03-9bee-6fd576d187f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
I installed the puppet module saz-rsyslog from puppet forge.I use The Foreman to configure nodes. The Foreman is used by puppet via configuration [master] "external_nodes" "/etc/puppet/node.rb"Since the saz-rsyslog module install, I have been receiving the following error off and on (not consistently) across many nodes on a puppet update (i.e. puppet agent -t):"Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Rsyslog] is already declared; cannot redeclare on node <node-name>"
My nodes are CentOS 5,6,7; and any various number of the nodes may experience this issue, but not all of them at the same time.One day I will see dozens of server with this error, and other nodes not having this issue. This may go on for days if I do not touch The Foreman.I'll make some changes to host configuration for puppet module class parameters in The Foreman - never the saz-rsyslog module though..After the changes, half or more of the servers having issue (not all) will magically have no problems.However, more nodes that did not have issues before, will now experience this issue.Also, this change of events is not directly related to The Foreman host configuration changes.I can simply perform a puppet module upgrade to a unrelated module (e.g. mine-yumconfig). After upgrading the unrelated module, again many nodes with this issue will now have it resolved, and different ones not experiencing the issue before will now begin experiencing it.The only clue I have is from this posting: http://grokbase.com/t/gg/puppet-users/165h0exgez/duplicate-resource-declaration-error"... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..."Can anyone help me understand this issue, or help me get it resolved permanently?When I search for answers, all I see are "You have written a duplicate class in your module."
RG, typically the Duplication declaration message is followed by some indicator of the file where it is duplicated and the line number. Are you receiving that information and maybe snipped it off the end of the message you included? An alternative might be any sort of `hiera_include()` or `create_resources()` calls whose arguments may include `rsyslog` more than once.
[Linux ~]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Rsyslog] is already declared; cannot redeclare on node server1.mydomain.test1.example.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
[Linux ~]#
On Wed, Oct 5, 2016 at 3:16 PM, <re-g...@wiu.edu> wrote:
I installed the puppet module saz-rsyslog from puppet forge.I use The Foreman to configure nodes. The Foreman is used by puppet via configuration [master] "external_nodes" "/etc/puppet/node.rb"Since the saz-rsyslog module install, I have been receiving the following error off and on (not consistently) across many nodes on a puppet update (i.e. puppet agent -t):"Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Rsyslog] is already declared; cannot redeclare on node <node-name>"My nodes are CentOS 5,6,7; and any various number of the nodes may experience this issue, but not all of them at the same time.One day I will see dozens of server with this error, and other nodes not having this issue. This may go on for days if I do not touch The Foreman.I'll make some changes to host configuration for puppet module class parameters in The Foreman - never the saz-rsyslog module though..After the changes, half or more of the servers having issue (not all) will magically have no problems.However, more nodes that did not have issues before, will now experience this issue.Also, this change of events is not directly related to The Foreman host configuration changes.I can simply perform a puppet module upgrade to a unrelated module (e.g. mine-yumconfig). After upgrading the unrelated module, again many nodes with this issue will now have it resolved, and different ones not experiencing the issue before will now begin experiencing it.The only clue I have is from this posting: http://grokbase.com/t/gg/puppet-users/165h0exgez/duplicate-resource-declaration-error"... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..."Can anyone help me understand this issue, or help me get it resolved permanently?When I search for answers, all I see are "You have written a duplicate class in your module." However, in my case, I did not write the saz-rsyslog module, I am only using it. It is a puppet-forge approved module with 635,000+ downloads. And without modifying the module, the issue can disappear, seemingly without rhyme or reason.-RG
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a81931c8-14b3-46d8-94ba-1f31c4c4453a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Can you undo the change in foreman, see if the problem goes away, then reimplement the change and see if the problem comes back? That would go a long way toward isolating the cause.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c5246478-278c-4d2b-ac0b-4e8986a45611%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
I understand the duplicate declaration issue - I have been reading about it for months trying to figure out this issue that keeps popping up. I am trying to figure out how it is possible I am getting a duplicate class declaration on a widely-used and approved module I have no control over (that no one else has reported any similar issue as mine), when using the forman to simply provide the values of parameters.
Maybe The Foreman is the cause of the ::rsyslog class to be declared more than once? Might you suggest this possibility?
But then I think, how would it? And why do I have this issue for hours on a server, and then at some magic moment the issue just disappears? Why with two of my servers sitting 1 IP away from each other, one will have this issue, and the other will not?
I consider that The Foreman is causing the problem, but then I should have the error on all hosts at the same time.
The only different in the YAML between hosts is the IP addresses... every other puppet module parameter value is exactly the same.
The Foreman has parameterized class support. It allows us to manage the values of the puppet module parameters.
Old Screen Shots with explainationWhen using the foreman, I am not writing any files, or declaring any classes. Instead, I manage the values of the puppet module parameters in the The Foreman WebUI, which is simply logic like 'if this host fact meets this criteria, then this puppet module parameter is set to this value'.
Well... The node I have been testing the duplicate declaration on uses a puppet secondary-master server, as it is on a remote network segment. It does not connect directly to the puppet primary-master in which The Forman is running on.So I did some work to get this particular "server1" node to use the puppet primary-master that The Foreman is running on. When I run a puppet update, it completes without error. When I switch back to the puppet secondary-master, I get the duplicate class error.They are both running puppet 3.8.7-1 on CentOS 6.The YAML produced by both is exactly 100% the same. So I can assume the YAML structure is not the issue.Would this suggest that the puppet secondary-master server is the issue, or the client connecting to it is perhaps not always getting what it wants from the slave?Remember that the puppet updates will complete correctly for many hours, then magically change to this error. And vice-versa, be in error for many hours, and then magically change to completing correctly. Also that sometime changing configuration in The Forman can trigger the Error to occur AND other times trigger to Error to stop occurring.Also note, I only have this problem with the saz-rsyslog module - NEVER with any other puppet module.When I remove the saz-rsyslog module, all issues disappear.
I have made sure the puppet modules are 100% in sync between primary and secondary master server.And I have restarted the puppet processes on the secondary-master server, but the error will continue on the nodes.
On Monday, October 10, 2016 at 9:07:35 AM UTC-5, jcbollinger wrote:
On Friday, October 7, 2016 at 10:39:58 AM UTC-5, re-g...@wiu.edu wrote:Well... The node I have been testing the duplicate declaration on uses a puppet secondary-master server, as it is on a remote network segment. It does not connect directly to the puppet primary-master in which The Forman is running on.So I did some work to get this particular "server1" node to use the puppet primary-master that The Foreman is running on. When I run a puppet update, it completes without error. When I switch back to the puppet secondary-master, I get the duplicate class error.They are both running puppet 3.8.7-1 on CentOS 6.The YAML produced by both is exactly 100% the same. So I can assume the YAML structure is not the issue.Would this suggest that the puppet secondary-master server is the issue, or the client connecting to it is perhaps not always getting what it wants from the slave?Remember that the puppet updates will complete correctly for many hours, then magically change to this error. And vice-versa, be in error for many hours, and then magically change to completing correctly. Also that sometime changing configuration in The Forman can trigger the Error to occur AND other times trigger to Error to stop occurring.Also note, I only have this problem with the saz-rsyslog module - NEVER with any other puppet module.When I remove the saz-rsyslog module, all issues disappear.
I am not prepared to believe that identical implementations of Puppet's catalog builder running on substantially identical platforms with identical inputs behave differently. Since you do see variations in behavior, therefore, I conclude that those differences can be attributed to differences in implementation, platform, or (most likely) inputs.
I have made sure the puppet modules are 100% in sync between primary and secondary master server.And I have restarted the puppet processes on the secondary-master server, but the error will continue on the nodes.
Those are good steps. To really troubleshoot this thoroughly, however, I think you'll need to be systematic: capture the ENC output for each catalog request for a given node (or for all nodes), with timestamps and associated success / failure of catalog compilation. Compare the ENC output for successful catalog builds with that for failed builds and look for systematics.
Either at the same time or separately, you should look into whether Puppet's environment cache has an impact here. Some of the behaviors you describe make me rather suspicious of this. Supposing that you are using directory environments, you should experiment both with disabling caching altogether (set the environment_timeout configuration option to 0 (its default)) and with caching indefinitely (set environment_timeout to "unlimited"). Note that Puppet recommends against using any other setting for that option. You could also try turning on the ignorecache option at the master.
JohnSo I ran a `puppet agent -t` on one of the problem nodes against the primary master puppet server (which was successful), and then afterwards the secondary master puppet server (which produces the duplicate declaration error for Class[Rsyslog]).The size of both catalog files are exactly the same (I am referring to this file: /var/lib/puppet/client_data/catalog/server1.mydomain.example.com.json ).The only difference inside the file is the order of items in the json arrays.
So the only difference I can see between the two puppet servers is the order of the overall elements in the catalogs' json hashes and arrays.Could this be a cause of the duplicate declaration error?
You were right, that catalog file (server1:/var/lib/puppet/client_data/catalog/server1.mydomain.example.com.json) was from a successful build/run. There is no new catalog file when the server1 node receives the duplicate declaration error.I did some further testing, and I hope I made a little progress on narrowing down the issue - perhaps this adds a clue:Summary:1. Both rsyslog and rsyslog::client are assigned to the node server1 via a class group in The Foreman2. When I remove the rsyslog::client assignment, the server1 node can perform a successful puppet update, without a duplicate declaration error
3. Then I added rsyslog::client assignment back into the group, I received the duplicate declaration error4. Then, one parameter at a time, I configured The Foreman to use the "default value" (checked the "use default" box) of each configured parameter for the node server1, when this fact matches: fqdn=server1.mydomain.example.com5. When all parameters are set to be "default value" for this node, even though the class is still assigned to the node via The Foreman, and even though the parameters are still set to a value for other nodes via The Foreman, the puppet update runs successfully without a duplicate declaration error on node server1
6. After this, my going back and setting just one parameter in the rsyslog::client class for this node causes the duplicate declaration error
Details:In The Foreman, I have a Puppet Class config group that I assign to the group of hosts in this remote location. I assign this same config group for every node in the network, even ones in the local network and other remote networks that do not experience duplicate class errors. Let me call this group DefG1.The DefG1 config group has my default puppet module classes defined in it, including the Rsyslog puppet module class.However, I am including a subclass rsyslog::client in addition to the main class rsyslog in this group DefG1. Because I need to configure the rsyslog client parameters for every node, namely the remote syslog server host.Now, this is not an issue with other puppet modules to include sub modules (for example I also use both foreman-puppet and foreman-puppet::config together without issue).
Nor has this been an issue for nodes performing puppet updates against the primary-master puppet server and all the other secondary-master puppet servers (I have 3 secondary-masters, 1 local network, and 3 remote networks with secondary-puppet servers).
I cannot configure the class parameters in The Forman in the puppet module's class for a node unless the puppet class is assigned to the node. I can assign the class (1) directly to the node, or (2) to the node group the node is in, or (3) to a class config group that I can assign to the node or the node group. I have done #3, the config group, and I assign the config group to the highest parent node group all nodes in all networks are a member of.So when any rsyslog::client class parameters are defined, I receive the duplicate declaration error for nodes only in this one remote network. Yesterday I was experiencing this duplicate declaration error in another one of my remote networks, but it resolved itself magically within 24 hours - no one touched The Foreman, and all I did was login and monitor the dashboard.
This is an issue with the format of the ENC YAML used between Foreman
and Puppet, and is best fixed in the module.
The list of classes is given as a hash/dictionary and so has no
particular order defined - it's down to the Puppet master/server to
iterate over it, and it probably does so in no particular order. It
isn't under Foreman's control.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/39fa6d99-eb37-4fb1-8e9c-7d04dcf0a574%40googlegroups.com.
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?If this is correct, then this is the root of the problem.
The next question is, why does Puppet assemble a catalog with classes out of order, when the ENC output has them in the correct order?
I have seen two answers to this question of classes getting out of order in the puppet catalog:
1. one class 'cA' depends on 'cB', so puppet puts 'cB' first in the catalog. - but this is not the reason in my case.2. "... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..." - which should be unlikely since it works on one secondary master and not the next secondary master, both configured exactly the same.
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?
Running `puppet agent -t` on the client against the primary master puppet server is successfulRunning `puppet agent -t` on the client against the secondary master puppet server fails with duplicate declaration
RG,I ultimately do not know the answer to your question (though I believe it may be due to the non-deterministic ordering when resources are not specifically ordered), however you should note that your primary service includes "rsyslog::client" and the secondary includes rsyslog::client (without the quotes). I am not sure if that is a red herring or if it means there is an actual difference in the delivered catalog and how puppet agent evaluates it, though it does seem odd that the ENC output has that slight variation between systems.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
It is useful to confirm that Foreman's ENC produces consistent output -- if indeed it does. I'm not surprised that you obtained identical ENC output from two different Foreman servers, but it's unclear to me whether you have compared the failure-case ENC output with the success-case ENC output. I am inclined to guess that these do differ, because clearly something changes.
Unless ....
Do check the the 'ordering' parameter in the [master] section of your master's puppet configuration. If it is specified, and the specified value is anything other than 'manifest', then it is possible that setting its value to 'manifest' (the default) will help. I emphasize, however, that this is a workaround for sloppy manifest code, not a bona fide solution.
You've made a lot of hay about saz-rsyslog being approved / known-good / high-quality, with no particular justification. Certainly the module being available from the Forge should not be interpreted to support such a conclusion. To be clear: having looked at the code of the module in question, I think it is generally of good quality. As I have already said, however, and as Dominic Cleal told you over on the Foreman group, the fundamental problem here is in the manifests -- the module's manifests. The module is designed and its manifests are written in a manner that is especially conducive to the kind of problem you've encountered.
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?If this is correct, then this is the root of the problem.
I'm not sure what significance you're attributing to those particular files, but it's probably the wrong significance. Yes, to the best of my knowledge and understanding, Puppet generates those files based on the node's certificate and facts, and the associated ENC output (if any). But no, there is no reason to think that the root of your problem lies here.
I have already explained the root of the problem several times, but I'll try just once more: the saz-rsyslog module is designed and implemented in a way that does not support declaring class ::rsyslog via a resource-like declaration (whether Puppet DSL's version of that or its ENC analog) while also declaring any of several other public classes from the same module. If there is any flaw to be attributed to Puppet here, it is that catalog building sometimes works for you anyway, not that it sometimes fails in the face this kind of misuse.
The next question is, why does Puppet assemble a catalog with classes out of order, when the ENC output has them in the correct order?
The order in which classes are listed in the ENC output is not significant, nor indeed is the order of entries in any YAML hash logically significant. That's basically the answer.
Depending on several underlying factors, it might be that under some circumstances you do reliably see classes specified by the ENC being evaluated in an order consistent with the lexical order of the ENC output. As I remarked in my previous message, the version of Ruby on which your master is running is one of my first guesses as to a source differences in that respect.
I have seen two answers to this question of classes getting out of order in the puppet catalog:
Classes are not getting out of order in the puppet catalog, nor indeed are the files that you earlier asked about (serialized) Puppet catalogs.
1. one class 'cA' depends on 'cB', so puppet puts 'cB' first in the catalog. - but this is not the reason in my case.2. "... If you do not see the error on every run then it is modulated by something that varies between runs. That could be almost anything: manifests, data, results of function calls, node facts, or ENC output. ..." - which should be unlikely since it works on one secondary master and not the next secondary master, both configured exactly the same.
You're now diverting to a different topic: why the observed behavior changes over time, as opposed to why it differs between masters. I have no reason to doubt your assertion that the manifests involved are the same everywhere. The data I refer to would be Puppet external data, which it seems you are not using, so that is trivially the same from run to run. But that still leaves many things that could vary over time; the list you quoted covers other likely candidates, but it was never intended to be exhaustive. If I had to guess, though, I'd place my bet on the ENC output varying. Do note that it doesn't have to be the part of the ENC output that designates classes rsyslog and rsyslog::client that changes; a variety of other changes might result in an inversion of the order in which those two are evaluated, too.
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?
There is an unsupported-use-of-module issue ultimately causing all your problems, as I have been saying since the beginning. It would be best to solve that problem, either by choosing a different module or by restricting yourself to supported uses of the module you have chosen (e.g. by using automated data binding to assign values to class parameters).
If you insist on continuing in your misuse and trying to work around the problems that follow from that, then it seems reasonable to focus on differences between the secondary master that fails and those that always work in this regard, if in fact they really do. Compare software versions (especially of Puppet and of the Ruby on which Puppet is running (and watch out for multiple Ruby versions being co-installed). Compare (puppet) config files. Heck, replace the troublesome secondary master with a clone of one of the others if you can't figure out what differs.
Running `puppet agent -t` on the client against the primary master puppet server is successfulRunning `puppet agent -t` on the client against the secondary master puppet server fails with duplicate declaration
I note in passing that you've now shifted back to the topic of behavior differing between different masters.
I assumed this because both the foreman and node files are created new every time server1 performs a puppet agent update, even when the result is a 400 error for the catalog compilation.If I am mistaken in this, then I need someone to tell me how to get the data of the catalog that is failing to be compiled, so that I can compare it with other sources.Here is the procedure I have been assuming (I am not an expert on puppet internals):1. (server1)puppet-agent -> puppet-master2. puppet-master -> (exec)puppet[external_nodes] server1.mydomain.example.com -> foreman ENC3. foreman ENC -> /var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml -> puppet-master4. puppet-master -> /var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml -> puppet-master(compile-catalog)5. puppet-master(compile-catalog) -> Error -> 400 Server Error -> (server1)puppet-agentI was assuming the node file is the data of the catalog to be compiled. I had noticed it is updated in every puppet agent run, even when the run is in error. And it was the differing piece between primary and secondary puppet masters.Please correct me.
Please let me know how to get the data of the catalog the puppet master is failing to compile for the puppet agent.
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?If this is correct, then this is the root of the problem.
I'm not sure what significance you're attributing to those particular files, but it's probably the wrong significance. Yes, to the best of my knowledge and understanding, Puppet generates those files based on the node's certificate and facts, and the associated ENC output (if any). But no, there is no reason to think that the root of your problem lies here.As I said earlier, I now assume I misunderstood this as being the file holding the data to be compiled as the catalog.I apologize for the misunderstanding, but that I intended it as the "root" of the catalog-compilation "problem". You are right that it is not the root of the entire underlying problem.Again, I need help to find the catalog data trying to be compiled, but which fails, if this /.../node/.. file is not that data.
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?
There is an unsupported-use-of-module issue ultimately causing all your problems, as I have been saying since the beginning. It would be best to solve that problem, either by choosing a different module or by restricting yourself to supported uses of the module you have chosen (e.g. by using automated data binding to assign values to class parameters)."restricting yourself to supported uses of the module"So using The Foreman as ENC has not been an official tested use of the module and thus is not supported? This is true for all modules on forge I have reviewed, except for the ones published by The Foreman group.To be supported, should I instead restrict my use of either declaring my classes in a site.pp or use hiera?
"using automated data binding to assign values to class parameters"What is automated data binding? Is that what I get when using hiera?
Would that mean I would forgo the use of The Forman to instead use hiera?
I have been assuming hiera was relatively the same as using The Foreman as ENC? Reference: https://ask.puppet.com/question/527/are-hiera-and-foreman-parameters-mutually-exclusive/
I am grasping at straws. I have been assuming what I can find different between the servers would reveal the issue.
I guess I need someone to tell me that if a puppet module is having the issue we describe in this thread, that due to the intended and expected nature of some function/process "X" in The Foreman or Puppet, I can expect to see the module's inheritance issue cause catalog-compilation errors at random times.
Example: it was suggested that the hash keys are not always ordered the same. So due to python sometimes ordering the hash keys differently, a module with inheritance issues like saz-rsyslog will sometimes cause a catalog compilation error, and sometimes not be an issue. This means Puppet expects modules to not have an inheritance issue like we are describing in order to guarantee there will not be issues during deployment, like in my case.
Package { allow_virtual => false }
Is this correct, that the yaml file in /var/lib/puppet/yaml/node/ is exclusively generated by Puppet based on the ENC yaml?If this is correct, then this is the root of the problem.
I'm not sure what significance you're attributing to those particular files, but it's probably the wrong significance. Yes, to the best of my knowledge and understanding, Puppet generates those files based on the node's certificate and facts, and the associated ENC output (if any). But no, there is no reason to think that the root of your problem lies here.As I said earlier, I now assume I misunderstood this as being the file holding the data to be compiled as the catalog.I apologize for the misunderstanding, but that I intended it as the "root" of the catalog-compilation "problem". You are right that it is not the root of the entire underlying problem.Again, I need help to find the catalog data trying to be compiled, but which fails, if this /.../node/.. file is not that data.
The starting point for classes to be applied is the combination of ENC output with the site manifest. I cannot speak to what may be in your site manifest. What file or files constitute your site manifest depends on your Puppet configuration. Some of the more likely possibilities are '<puppet root>/manifests/site.pp', '<puppet root>/environments/production/manifests/site.pp', or jointly all files of the form '<puppet root>/environments/production/manifests/*.pp'. Within the site manifest, declarations at top scope and within the node block best matching the target machine (if any match at all) are relevant.
Is there a configuration issue causing this issue? Where would I begin looking for something that may be causing this puppet-catalog problem?
There is an unsupported-use-of-module issue ultimately causing all your problems, as I have been saying since the beginning. It would be best to solve that problem, either by choosing a different module or by restricting yourself to supported uses of the module you have chosen (e.g. by using automated data binding to assign values to class parameters)."restricting yourself to supported uses of the module"So using The Foreman as ENC has not been an official tested use of the module and thus is not supported? This is true for all modules on forge I have reviewed, except for the ones published by The Foreman group.To be supported, should I instead restrict my use of either declaring my classes in a site.pp or use hiera?
Puppet supports declaring classes via an ENC generally. Inasmuch as its implementation violates the DSL's documented constraints, however, the 'rsyslog::client' class is not supported at all in the sense of the term I was using.
Since avoiding that class is not a viable solution (unless by choosing an altogether different module), the next best thing would be to stick to uses that are well known to work despite not actually being supported -- in this case, that would be to avoid using (the ENC analog of) a resource-like declaration for the base class, rsyslog. You have exactly one option for customizing that class's parameters without modifying it or using a resource-like declaration: automated data binding.
"using automated data binding to assign values to class parameters"What is automated data binding? Is that what I get when using hiera?
It is documented here: https://docs.puppet.com/hiera/3.2/puppet.html#automatic-parameter-lookup. It is based on Hiera, but it would not be quite correct to say it's "what I get when using hiera", because Hiera supports other uses as well.
Would that mean I would forgo the use of The Forman to instead use hiera?
It would mean that you forgo using The Foreman for setting the parameters of class rsyslog, at least for those nodes to which you also assign another class from the same module, such as rsyslog::client. You should otherwise be able to continue using The Foreman as you currently do.
I have been assuming hiera was relatively the same as using The Foreman as ENC? Reference: https://ask.puppet.com/question/527/are-hiera-and-foreman-parameters-mutually-exclusive/
No, they are not the same thing at all, though they can serve many of the same purposes (and the answer you linked says pretty much the same thing). Hiera is not an ENC, and Puppet does not use it in the same way or via the same interface that it uses an ENC. Binding values to class parameters via (hiera-supported) automatic data binding has different semantics than does binding parameter values via an ENC. From a Foreman perspective, you might view automated data binding as affecting the default values for class parameters.
I am grasping at straws. I have been assuming what I can find different between the servers would reveal the issue.
Indeed, the problems are mysterious, and it is entirely reasonable to suppose that differing behavior arises from discoverable differences between machines. Myself, I had high hopes for an explanation in terms of differing versions of Ruby. I guess there's still a chance that a difference in the installed Ruby modules might factor in. And I wasn't kidding when I suggested replacing the machine on which the misbehavior is observed with a clone of one of those that behave as you want.
However, I also hold out some suspicion that the problem may not be in the machine at all, but rather in the inputs (ENC output varying over time; differences in other parts of the ENC output; node facts). Or the difference may be in the dynamic operating environment, including Puppet runtime state. I'm not sure how to test that directly, but it might be interesting to swap masters to see whether the problem sticks with the machine or with the workload.
I guess I need someone to tell me that if a puppet module is having the issue we describe in this thread, that due to the intended and expected nature of some function/process "X" in The Foreman or Puppet, I can expect to see the module's inheritance issue cause catalog-compilation errors at random times.Example: it was suggested that the hash keys are not always ordered the same. So due to python sometimes ordering the hash keys differently, a module with inheritance issues like saz-rsyslog will sometimes cause a catalog compilation error, and sometimes not be an issue. This means Puppet expects modules to not have an inheritance issue like we are describing in order to guarantee there will not be issues during deployment, like in my case.
Well, the problem with hashes is that hash iteration order is not necessarily predictable in advance. However, given the same hash implementation, initialized the same way, and subjected to the same sequence of additions and removals, you should see the same iteration order on every trial. On the other hand, if the sequence of additions and removals changes, and especially if more, fewer, or different keys are added in different trials, then you cannot rely on iteration order to be consistent, even in part. That's one reason why I earlier pointed out that the problem does not have to be in the portion of the ENC output that pertains directly to the rsyslog classes.
So no, I cannot tell you that with everything remaining the same, you should expect that Puppet's catalog building behavior might vary from run to run. Quite the opposite. But on the other hand, you can never have everything exactly the same.
I sympathize with you in your difficulty discovering the specific difference(s) that explain the variation in outcome, and some of my suggestions have been aimed at helping you there. But you should not assume that figuring it out will lead you to a solution you like better than those I have proposed. It might do. It also might not. And I cannot evaluate for you whether it's worthwhile to keep looking.
John
class { 'rsyslog':
preserve_fqdn => true,
gnutls_package_name => false,
relp_package_name => false,
rsyslog_package_name => false
}
As it seems saz-rsyslog is the best puppet module for managing rsyslog, I decided to take a hint from @jcbollinger on configuration, rather than replacing the module.I decided to change the values of the parent Rsyslog class parameters so that they are the seen as the "default" for The Foreman (I refer to "default" as eluded to in a previous reply regarding use of Hiera to change these values). But I did not want to implement Hiera just for one class and four parameters. So I added the following changes on all puppet master servers to the production/manifests/site.pp file:File: /etc/puppet/environments/production/manifests/site.pp
class { 'rsyslog':
preserve_fqdn => true,
gnutls_package_name => false,
relp_package_name => false,
rsyslog_package_name => false
}This changes the values of these parameters globally, which is how I had it set in The Foreman any way.And in The Foreman, I removed all values set for the parent Rsyslog class parameters, leaving them checked to use the default value. I left the values set in The Foreman for Rsyslog::client subclass parameters unchanged (which is necessary because these values change based on node facts). I did leave the rsyslog parent class assigned to all nodes in The Foreman, though the ENC output does not set any parameters for this class.This fixed the duplicate declaration issue I was experiencing.The reason why I believe this fixed the duplicate declaration error is because I assume Puppet's intended behavior is to always declare in the catalog the classes from the site.pp file before the classes from the ENC.This is obviously not ideal, but I justify this with the argument that I am simply changing the "default" values of the saz-rsyslog puppet module Rsyslog class - but without having to modify the module's code.This is a good enough work-around for me for now.-RG