dict merge - with_dict expects a dict

[John McNulty]

Hi,

So here's the scenario - I'm given a data structure that when converted to yaml looks like this (names & numbers invented).

host_routes:
  route-oob:
    device: bond.123
    routes:
      10.118.0.0/16: '10.118.255.254'
  route-int:
    device: bond.456
    routes:
      172.168.0.0/23: '172.168.1.254'

I can use that with a template to create the network-script route files needed, that's easy enough.   But in real life there can be up to several OOB routes that this host will need depending on which datacentre / hall / security zone it resides in. e.g. 

standard_routes:
  route-oob:
    device: bond.123
    routes:
      10.198.0.0/24: '10.118.255.254'

I'd like to add these additional routes to the appropriate group_var file and then merge them together when creating the route files.   I can't get this to work though.  I've tried all the following:

with_dict: host_routes|union(standard_routes)

with_dict: union(host_routes, standard_routes)

with_dict: host_routes + standard_routes

and get the same error every time ..

fatal: [localhost] => with_dict expects a dict

Is there another filter or some other way of doing this?   I don't want to change the default hash_behaviour to merge just for this one use case as that has much wider implications.

Thanks,  John

[John McNulty]

So the reason why I tried using union was this github issue ( https://github.com/ansible/ansible/issues/7495 ) which implied it should work (assuming the PR got actioned).

But looking at the code ( lib/ansible/runner/filter_plugins/mathstuff.py ) it doesn't look like that's the case as union() doesn't match what this article (http://stackoverflow.com/questions/38987/how-can-i-merge-two-python-dictionaries-in-a-single-expression) describes as the 'pythonic way'.

z = x.copy()

z.update(y)

So I tried to emulate that with ...

with_dict: "{% set r = host_routes.copy() %}{% r.update(standard_routes) %}{{ r }}"

resulting in ..

fatal: [localhost] => template error while templating string: unknown tag 'r'

I've also scanned all the Jinja2 filters ( http://jinja.pocoo.org/docs/dev/templates ) looking for a potential candidate and can't see anything that looks suitable.  

I really am an Ansible & jinja2 pseudo-novice, so I'm kind of stumbling around a bit here :)

[Jonathan Davila]

Would you mind sharing the full body of the task that pertains to this as well as the template file?

[Jonathan Davila]

so I've got a working sample for you. This is the exact test I did.

---
- name: Template Test For Route File
  connection: local
  hosts: localhost
  vars:

   host_routes:
     route-oob:
        device: bond.123
        routes:
          10.118.0.0/16: '10.118.255.254'
     route-int:
        device: bond.456
        routes:
          172.168.0.0/23: '172.168.1.254'

   standard_routes:
     route-oob:
        device: bond.123
        routes:
          10.198.0.0/24: '10.118.255.254'

  tasks:
    - template: src=t.j2 dest=./templated

 

and my jinja template file looks like:

{% if host_routes %}
{% for k in host_routes %}
{% for route in host_routes[k]['routes'] %} {{ route }} via {{host_routes[k]['routes'][route]}} dev {{ host_routes[k]['device'] }}
{% endfor %}
{% endfor %}
{% endif %}
{% if standard_routes %}
{% for k in standard_routes %}
{% for route in standard_routes[k]['routes'] %} {{ route }} via {{standard_routes[k]['routes'][route]}} dev {{ standard_routes[k]['device'] }}
{% endfor %}
{% endfor %}
{% endif %}

which should then output to a file called templated in your CWD

 172.168.0.0/23 via 172.168.1.254 dev bond.456
 10.118.0.0/16 via 10.118.255.254 dev bond.123
 10.198.0.0/24 via 10.118.255.254 dev bond.123

Actually don't need the with_dict loop here at all.

Hopefully that helps, let us know otherwise.

[John McNulty]

Hi,

For just processing host_routes I had.

- name: Write network routes
  template: >
    src=route.j2
    dest={{ net_path}}/route-{{ item.value.device }}
    mode=0644 owner=root group=root
  with_dict: host_routes
  when: item.value.routes is defined
  tags: netdev

And the template was ..

{% for network,gateway in item.value.routes.iteritems() %}
{{ network }} via {{ gateway }} dev {{ item.value.device }}
{% endfor %}

I needed the 'when' conditional because the dict sometimes contains something like this instead to ensure that old/retired/changed network-script route files must be removed:

  routes-ext:
    device: 'eth6.111'
    absent: true
  routes-other:
    device: 'eth6.222'
    absent: true

And I cater for that with ..

- name: Remove 'absent' route files
  file: path={{ net_path}}/route-{{ item.value.device }} state=absent
  with_dict: host_routes
  when: item.value.absent is defined
  tags: netdev

[Jonathan Davila]

Does my example above help you as far as the templating piece? 

Personally I would abstract out the desired state for the route-file status into it's own dict to smth like this:

route_file_state:
   routes-other: present
   routes-ext: absent
   

would prevent that extra attribute from interfering with my jinja example and needing to add more logic.

[John McNulty]

Hi again.   Thanks .. I got so hung up on using with_dict I didn't consider taking another tack.

I'm reconsidering the decision to not to use  hash_behaviour=merge as that solves my problem + would also come in handy elsewhere, like combining default sysctl parameter changes with additional application & environment requirements.    If I don't then it would be an interesting exercise for me to write a custom module to merge dictionaries, or even add a new ansible filter.  

Cheers,  J

[John McNulty]

Yes, I can work with that.

Thanks for your help.