Merge Dictionary

[Isa]

Hi All

How can I merge dictionary with the same name.

Example:

groups_vars/all

users:
  - name: jsmith
    comment: 'John Smith'
    shell: /bin/bash
    groups: wheel



group_vars/postgresql.yml

users:
  - name: tdoe
    comment: 'Tom Doe'
    shell: /bin/bash
    groups: wheel



host_vars/server1

users:
  - name: iwhite
    comment: 'Ian White'
    shell: /bin/bash
    groups: wheel

The usecase for this is some users should only have access to a few servers while other users have access to all etc. I also dont want to duplicate users in many yaml files.

If the above dictionary was merged then if my inventory looks like this

[postgres]

sererp

[haproxy]
server1
[nonprod]

serverd

jsmith will haev access to serverp/1/d

tdoe will have access to serverp

iwhite will have access to server1

I have found similar question on the internet, but most were a few years old. I hoping something has changed.

[Vladimir Botka]

On Fri, 22 May 2020 04:23:51 -0700 (PDT)
Isa <misa...@gmail.com> wrote:

> How can I merge dictionary with the same name.
>

> *groups_vars/all*

> users:
> - name: jsmith
> comment: 'John Smith'
> shell: /bin/bash
> groups: wheel
>
>

> *group_vars/postgresql.yml*

> users:
> - name: tdoe
> comment: 'Tom Doe'
> shell: /bin/bash
> groups: wheel
>
>

> *host_vars/server1*

> users:
> - name: iwhite
> comment: 'Ian White'
> shell: /bin/bash
> groups: wheel

The merging of the dictionaries would require unique keys. Instead, it's
possible to concatenate a list of dictionaries. For example

- set_fact:
my_users: "{{ my_users|default([]) + my_vars.users }}"
loop:
- group_vars/all
- group_vars/postgresql.yml
- host_vars/server1
vars:
my_vars: "{{ lookup('file', item)|from_yaml }}"

HTH,

-vlado

[Isa]

Thanks that works , I wish it was cleaner though. Appreciate the help

[Vladimir Botka]

On Fri, 22 May 2020 09:58:10 -0700 (PDT)
Isa <misa...@gmail.com> wrote:

> Thanks that works , I wish it was cleaner though. Appreciate the help

> >

> > - set_fact:
> > my_users: "{{ my_users|default([]) + my_vars.users }}"
> > loop:
> > - group_vars/all
> > - group_vars/postgresql.yml
> > - host_vars/server1
> > vars:
> > my_vars: "{{ lookup('file', item)|from_yaml }}"

You're welcome. Just curious what might be cleaner? Could you describe your
idea in any language, or meta perhaps?

[Vladimir Botka]

(list in the loop)

On Fri, 22 May 2020 23:58:06 +0200
Vladimir Botka <vbo...@gmail.com> wrote:

> > > > > - set_fact:
> > > > > my_users: "{{ my_users|default([]) + my_vars.users }}"
> > > > > loop:
> > > > > - group_vars/all
> > > > > - group_vars/postgresql.yml
> > > > > - host_vars/server1
> > > > > vars:
> > > > > my_vars: "{{ lookup('file', item)|from_yaml }}"

> > > > Thanks that works , I wish it was cleaner though. Appreciate the help

> > > You're welcome. Just curious what might be cleaner? Could you describe your
> > > idea in any language, or meta perhaps?

> > In puppet using hiera you can use hiera hash which merges the matches.
> > It's useful in many cases another example would be if you managing
> > iptables. You would want one rule for all servers example allow port 22 and
> > and other rules based on type of server example 5432 for PostgreSQL. I
> > don't want to duplicate port 22 in all my server types. One reason for this
> > if someone added a new server type they might miss a port. Hope this make
> > sense :)
> > https://ask.puppet.com/question/13592/when-to-use-hiera-hiera_array-and-hiera_hash/

> There is hiera lookup plugin in Ansible.
> https://docs.ansible.com/ansible/latest/plugins/lookup/hiera.html#hiera-get-info-from-hiera-data
> Would it be possible to post the [mcve] Puppet code of your example? At gist
> perhaps? I'd like to try and translate it to Ansible with the hiera plugin,
> and to see what's cleaner. Hope this makes also sense :)
> [mcve] https://stackoverflow.com/help/minimal-reproducible-example

[Shaun Rampersad]

On Saturday, 23 May 2020 00:06:10 UTC+2, Vladimir Botka wrote:

> There is hiera lookup plugin in Ansible.
> https://docs.ansible.com/ansible/latest/plugins/lookup/hiera.html#hiera-get-info-from-hiera-data
> Would it be possible to post the [mcve] Puppet code of your example? At gist
> perhaps? I'd like to try and translate it to Ansible with the hiera plugin,
> and to see what's cleaner. Hope this makes also sense :)
> [mcve] https://stackoverflow.com/help/minimal-reproducible-example

An Simple example from Puppet that we have working. We have a firewall manifest that applies rules based on a hash. We simply lookup all hashes and create resources.

Puppet Code

      $fhl_firewall = hiera_hash('profile::base::firewall', undef)

      if $fhl_firewall != undef {

        create_resources(firewall, $fhl_firewall)

      }

Heira layout:

├── common.yaml

├── internal

│   ├── nonprod

│   │   ├── elasticsearch.yaml

│   │   └── logstash.yaml

common.yaml:

profile::base::firewall:

  '100 allow Prometheus node_exporter':

    proto:

      - 'tcp'

    dport:

      - '9100'

    action: 'accept'

internal/nonprod/elasticsearch.yaml

profile::base::firewall: 

 '100 allow 9200 from host1':

    proto: 'tcp'

    dport: '9200'

    action: 'accept'

    source: '192.168.101.15/32'

 '100 allow 9200 from host2':

    proto: 'tcp'

    dport: '9200'

    action: 'accept'

    source: '192.168.101.16/32'

internal/nonprod/logstash.yaml

profile::base::firewall:

  '100 allow kibana':

    proto: 'tcp'

    dport: '5601'

    action:  'accept'

  '100 allow log_forwarder':

    proto:  'tcp'

    dport:

    - '10510-10520'

    action:  'accept'

Hope this makes sense. 

PS: I'm part of Isa's team :) I'll also take a look at the ansible plugin

 

[Vladimir Botka]

On Sun, 24 May 2020 07:10:36 -0700 (PDT)
Shaun Rampersad <shau...@gmail.com> wrote:

> PS: I'm part of Isa's team :) I'll also take a look at the ansible plugin

Let me know if this might meet the high standards of the code.