Hash iteration order in a template not consistent
Martijn Grendelman
I did some basic googling, but didn't find an answer yet. I am sorry if
this is a FAQ.
In a manifest for creating an Apache config file, I define a hash like this:
$aliases = {
'/foo/' => '/home/foo/www/',
'/bar/' => '/home/bar/www/',
'/baz/' => '/home/baz/www//'
}
Then, in a template, I have:
<% aliases.each_pair do |key, val| -%>
Alias <%= key %> <%= val %>
<% end -%>
The result is mostly what I expect, but every once in a while, the order
in which the Aliases are generated from the 'each_pair' loop changes,
resulting in a different file.
The Ruby docs state that "hashes enumerate their values in the order that
the corresponding keys were inserted.", but is that not true for Puppet
hashes?
I did stumble across this post:
http://serverfault.com/questions/368784/puppet-and-templates-how-to-loop-sequently-and-not-randomly
which suggests to do something like
<% aliases.sort_by {|key, value| key}.each_pair do |key, val| -%>
<% end -%>
Will it work? Is that a proper solution?
Thanks!
Martijn Grendelman
R.I.Pienaar
----- Original Message -----
> From: "Martijn Grendelman" <mar...@iphion.nl>
<snip>
> http://serverfault.com/questions/368784/puppet-and-templates-how-to-loop-sequently-and-not-randomly
>
> which suggests to do something like
>
> <% aliases.sort_by {|key, value| key}.each_pair do |key, val| -%>
> <% end -%>
>
> Will it work? Is that a proper solution?
ruby hashes are not stored in predictable order so this will happen, the proposed
solution should work.
But as always the best is just to test it and see how it goes, it wont bite :)
Sergey Zhuga
<% aliases.sort.each do |alias| -%>
Alias <%= alias.first %> <%= alias.last %>
<% end -%>
Regards.
Martijn Grendelman
<% aliases.sort_by {|key, value| key}.each do |key, val| -%>
seems to do the trick. 'each_pair' doesn't work here, because the sort_by
returns an array. Again, I learned something :-)
Thanks,
Martijn.
Matthias Saou
Martijn Grendelman <mar...@iphion.nl> wrote:
[...]
> >
> > ruby hashes are not stored in predictable order so this will
> > happen, the proposed solution should work.
> >
> > But as always the best is just to test it and see how it goes, it
> > wont bite :)
>
> <% aliases.sort_by {|key, value| key}.each do |key, val| -%>
>
> seems to do the trick. 'each_pair' doesn't work here, because the
> sort_by returns an array. Again, I learned something :-)
...and what about those of us which want the hash entries to appear in
the exact same order they are present in the puppet manifest? From what
I've seen, it was working that way up to 2.6 included, and only gets
"randomized" with puppet 2.7.
I've been doing things like this for a while now :
mm_cfg_settings => {
'ALLOW_SITE_ADMIN_COOKIES' => "Yes",
'PUBLIC_ARCHIVE_URL' => "'https://%(hostname)s/pipermail/%(listname)s'",
'MTA' => "'Postfix'",
'POSTFIX_STYLE_VIRTUAL_DOMAINS' => "'False'",
'DEFAULT_SUBJECT_PREFIX' => "''",
'DEFAULT_REPLY_GOES_TO_LIST' => "1",
},
<% mm_cfg_settings.each do |key,value| -%>
<%= key %> = <%= value %>
<% end -%>
In this particular example, order isn't critical other than for
readability, but I have some others where items must be in the same
order as they appear in the manifest's hash or things will break.
Is there a way to keep using hashes if the order from the manifest must
be kept in a file generated from the template?
Matthias
Andre Nathan
jcbollinger
On Wednesday, February 19, 2014 1:47:35 PM UTC-6, Andre Nathan wrote:
HelloSorry to ressurect this old thread, but I've just found this issue upgrading from Puppet 2.7.x to Puppet 3. It's true that ruby 1.8 hash order cannot be relied on, but it should always be the same, right? I mean, one doesn't know which order the hash contents will be iterated on, but whatever order ruby chooses should never change.
The iteration order of an existing hash should not change as long as entries are neither added nor removed. Probably, a new hash created via the same sequence of pair insertions and deletions, using the same hash implementation, will also have the same iteration order.
We had this working with no issues in 2.7.x and ruby 1.8, but now on Puppet 3 we're getting random reorderings. I suspect there's a problem in Puppet in this case.
File a ticket if you wish, but personally, I'm inclined to say that any reliance on iteration order of a hash is dodgy. If iteration order matters then you need to take proactive measures to ensure that you reliably get the (an) order that works for you. Puppet does not document the iteration order of its hashes, or their Ruby representation in templates, so it is irrelevant that what you were doing worked in your particular prior context.
For what it's worth, I speculate that in Puppet 2.7, templates got a reference to the actual Ruby hash backing the corresponding puppet variable, whereas in Puppet 3 they get copies. That would allow Puppet to enforce its invariant that variables' values never change once assigned, which at one time could be broken via templates. Some reasonable implementations of hash copying could end up inserting entries into the copy in a different order than they were inserted into the original, thus giving the copy a different iteration order despite having the same contents.
John
Andre Nathan
File a ticket if you wish, but personally, I'm inclined to say that any reliance on iteration order of a hash is dodgy. If iteration order matters then you need to take proactive measures to ensure that you reliably get the (an) order that works for you.
Andre Nathan
jcbollinger
I understand that you had an unpleasant surprise here. I also understand the nature of the problem with respect to template implementation. I sympathize, but a change in undocumented behavior is not a bug. For correctness with respect to file changes, it was always necessary for your templates to ensure some specific iteration order, even though the particular one chosen doesn't matter to the file consumer.
Who knows, though, PL might nevertheless accept your ticket and provide the feature you request.
John