Merge/Combine Inventory Variables with Defaults rather than override

[christoph....@netresearch.de]

Hey folks,

the defaults/main.yml in a Role will be overridden in the precedence chain. In essence, as I understand, it is a dictionary that will be overlayed/merged/combined with sub-sequent variable settings (e.g. by inventory variables) in the manner as described here. But I suspect, this isn't done recursively.

How can I achieve recursive combination of dictionaries in the defaults?

I discovered that many definitions are flat, e.g.

myapp_component1_feature1: defaultA
myapp_component1_feature2: defaultB
myapp_component2_feature1: defaultC
myapp_component2_feature2: defaultD

instead of

myapp:
  component1:
    feature1: defaultA
    feature2: defaultB
  component2:
    feature1: defaultC
    feature2: defaultD

The second schema is favorable not only for elegance but also if e.g. I would like to iterate over components of myapp or over features of a component with Jinja. However, when I want to use the defaults in e.g. an inventory file in a suggestive manner

www.example.com
  vars:
    myapp:
      component1:
        feature2: myvalueB
      component2:
        feature1: myvalueC

The sub-dictionaries will be overridden instead of being combined. It will only contain the explicit elements.

[Kai Stian Olstad]

On 04.06.2018 15:37, christoph....@netresearch.de wrote:
> the defaults/main.yml in a Role will be overridden in the precedence
> chain.
> In essence, as I understand, it is a dictionary that will be
> overlayed/merged/combined with sub-sequent variable settings (e.g. by
> inventory variables) in the manner as described here

> <https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#combining-hashes-dictionaries>.

> But I suspect, this isn't done recursively.

That link to is a filter that you can use in your code to combine
hashes/dictonaries and has nothing to on how Ansible overwrite/replace
variables.

Default Ansible overwrite the whole variables in a predetermine order,
variable precedence, that is documented here
http://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable

Ansible has an option hash_behavior(default is replace) that you can set
to merge, it will then merge hashes/dictonaries instead of replacing
them.

--
Kai Stian Olstad

[christoph....@netresearch.de]

Hi Kai,

 

Ansible has an option hash_behavior(default is replace) that you can set
to merge, it will then merge hashes/dictonaries instead of replacing
them.

Unfortunately, DEFAULT_HASH_BEHAVIOUR (Ansible Docs) states that it is not a recommended practice. But luckely, I found a way that seems to be pretty straight-forward:

role/default/main.yml

myapp_defaults:

  component1:
    feature1: defaultA
    feature2: defaultB
  component2:
    feature1: defaultC
    feature2: defaultD

role/vars/main.yml

myapp_vars: |
  {%- if myapp is defined and myapp is iterable -%}
  {{ myapp_defaults | combine (myapp, recursive=True) }}
  {%- else -%}
  {{ myapp_defaults }}
  {%- endif -%}

The usage in the role/tasks/xyz.yml then is e.g.

- name: Some task
  XXX:
    YYY: "{{ myapp_vars.component2.feature1 }}"
    ...
...

Applying the role e.g. with play vars

# My Playbook

- name: setup MyApp
  vars:
    myapp:
      component2:
        feature1: "foo"
        feature2: "bar"
  tasks:
  - include_role:
      name: myapp

Variable precedence is important, e.g. the following wouldn't work:

# My Playbook

- name: setup MyApp
  tasks:
  - include_role:
      name: myapp
    vars:
      myapp:
        component2: ...
      ...

since role/vars/main.yml is already read in when include_vars are applied.

[Max Bigras]

Hi christoph,

Was very excited to see your solution to this problem.

I've been struggling to find a way to work around ansible replacing dictionaries.

Are you still using this approach?

How has it been working for you?

Have you made any changes?

[Aravinda. R]

Hi Christoph,

I tried this with one of my Ansible work, looks like it does not work well if we have a nested dict. any suggestions ?