Sort by IP in ERB (related to puppetlabs-haproxy and MODULES-1919)

[Tom Limoncelli]

The puppetlabs-haproxy module has a minor annoyance where by the
"bind" statements are sorted lexicographically instead of by IP
address. (Full description here:
https://tickets.puppetlabs.com/browse/MODULES-1919)

My attempt to fix this bug was to change the ERB template:

diff --git a/templates/fragments/_bind.erb b/templates/fragments/_bind.erb
index e60983a..a04d021 100644
--- a/templates/fragments/_bind.erb
+++ b/templates/fragments/_bind.erb
@@ -1,6 +1,6 @@
<% require 'ipaddr' -%>
<%- if @bind -%>
-<%- @bind.sort.map do |address_port, bind_params| -%>
+<%- @bind.sort_by { |address_port, bind_params|
address_port.split('.').map{ |octet| octet.to_i} }.map do
|address_port, bind_params| -%>
bind <%= address_port -%> <%= Array(bind_params).join(" ") %>
<%- end -%>
<%- else -%>

This works. However, the results are slightly different on old
versions of Ruby. If you look at the TravisCI output, you'll see
slightly different results for Ruby 1.8.7. It looks like something
changed in Ruby 1.9.

https://travis-ci.org/puppetlabs/puppetlabs-haproxy/builds/57502529

I don't have a lot of deep Ruby knowledge. Can anyone suggest either a
way to fix the code or the test?

Thanks!
Tom

--
Email: t...@whatexit.org Work: tlimo...@StackOverflow.com
Skype: YesThatTom
Blog: http://EverythingSysadmin.com

[jcbollinger]

On Tuesday, April 7, 2015 at 11:09:59 AM UTC-5, Tom Limoncelli wrote:

 

 If you look at the TravisCI output, you'll see
slightly different results for Ruby 1.8.7.  It looks like something
changed in Ruby 1.9.

https://travis-ci.org/puppetlabs/puppetlabs-haproxy/builds/57502529

I don't have a lot of deep Ruby knowledge. Can anyone suggest either a
way to fix the code or the test?

Are you saying that all the tests pass on Ruby 1.9, but some fail on Ruby 1.8.7?

I'm not sufficiently familiar with the module to quite understand the details of the test failures, but it looks like the sort order you are trying to apply does not very effectively cover the space of keys it may be used to sort.  In particular, what order is supposed to be used when the bind_address does not start with a dotted-quad IP address?  Some order will fall out from your code, but there's plenty of room for implementation-defined behavior there.


John

[Tom Limoncelli]

On Wed, Apr 8, 2015 at 9:21 AM, jcbollinger <John.Bo...@stjude.org> wrote:
> Are you saying that all the tests pass on Ruby 1.9, but some fail on Ruby
> 1.8.7?
>
> I'm not sufficiently familiar with the module to quite understand the
> details of the test failures, but it looks like the sort order you are
> trying to apply does not very effectively cover the space of keys it may be
> used to sort. In particular, what order is supposed to be used when the
> bind_address does not start with a dotted-quad IP address? Some order will
> fall out from your code, but there's plenty of room for
> implementation-defined behavior there.

John,

Yes, that's exactly the problem. The addresses that don't start with
dotted-quad IP addresses "fall out" one way in Ruby 1.8.7 and a
different order in Ruby 1.9+. Each is stable, but different.

[jcbollinger]

On Friday, April 10, 2015 at 1:50:31 PM UTC-5, Tom Limoncelli wrote:

On Wed, Apr 8, 2015 at 9:21 AM, jcbollinger <John.Bo...@stjude.org> wrote:
> Are you saying that all the tests pass on Ruby 1.9, but some fail on Ruby
> 1.8.7?
>
> I'm not sufficiently familiar with the module to quite understand the
> details of the test failures, but it looks like the sort order you are
> trying to apply does not very effectively cover the space of keys it may be
> used to sort.  In particular, what order is supposed to be used when the
> bind_address does not start with a dotted-quad IP address?  Some order will
> fall out from your code, but there's plenty of room for
> implementation-defined behavior there.

John,

Yes, that's exactly the problem.  The addresses that don't start with
dotted-quad IP addresses "fall out" one way in Ruby 1.8.7 and a
different order in Ruby 1.9+.  Each is stable, but different.

So you understand the problem, and you seem to understand most of the needed elements of Ruby well enough to use them to solve the problem.  I guess you're asking for some Ruby-fu to pull it all together.

You're already generating a compound search key (although it's not strictly necessary for what you're doing so far); a reasonable solution would be a compound sort key that captures the IP address (if any) as the first part, and the rest of the hash key as a second part.  Here's a way to approach it with the help of a Ruby regex:

<%
require 'ipaddr'
if @bind
  @bind.sort_by { |address_port, bind_params|
    md = /^((\d+)\.(\d+)\.(\d+)\.(\d+))?(.*)/.match(address_port)
    [ (md[1] ? md[2..5].inject(0){ |addr, octet| (addr << 8) + octet.to_i } : -1), md[6] ]

  }.map do |address_port, bind_params|
-%>
   bind <%= address_port -%> <%= Array(bind_params).join(" ") %>
<%

  end
else
-%>

In the sort_by() block, variable 'md' captures the match data for matching the given regex against the 'address_port' string.  The regex will match any string; the important part is the capturing groups.  If the string starts with a dotted-quad address then the whole address is captured as group 1, and the segments are captured as groups 2 - 5.  Whatever is not matched as a dotted-quad address is always captured as group 6.  The dotted-quad match is all-or-nothing; either four segments will be captured or none.  The sort key is then formed as a two-element array: the first element is -1 if no address is given, else the 4-byte integer IP address, and the second element is the string tail.


John

[jcbollinger]

On Monday, April 13, 2015 at 10:10:52 AM UTC-5, jcbollinger wrote:

The sort key is then formed as a two-element array: the first element is -1 if no address is given, else the 4-byte integer IP address, and the second element is the string tail.

Note that the important thing for consistency is that whole hash key appears in one form or another in the sort key.

Note, too, that this scheme could be extended fairly easily to sort numerically by (starting) port number as well.  The version given will sort lexicographically by the tail, so you would get this relative order, for instance:

1.2.3.4:8000 blah
1.2.3.4:81 blah blah
1.2.3.4:820-821 waffle


John

[Tom Limoncelli]

John,

That looks great! I'll give that a try. (Though possibly after a
delay... I have some higher priority issues on my plate)