Hierarchical overrides for playbooks

[Javier Domingo Cansino]

Hello,

First of all, I am coming from puppet. I have already done some experiments with ansible, and created a user management module, mainly to create the root Authkeys file. This was a little approach to see how overrides etc are handled by ansible.

The result was having the module executed twice, one for the general case and the other for the specific one. Because of this, I wondered whether it would be possible to have overrides correctly done.

I have read a thread asking for Ansible's hiera, and for what I understood, you proposed using lookups and external inventories. I see ansible has a different way to do stuff, but I don't understand how this sort of features would provide such flexibility.

I would be glad if someone could help me understand how to correctly structure Ansible code for my use case.

[Serge van Ginderachter]

Hi Javier,

Perhaps you could start with explainig - for non-puppeteers here - what an 'override' exactly is, and show some basic high level example of what you try to accomplish?

Serge

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/926c5154-0a4e-4a96-95c2-9ecf09ba95f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Javier Domingo Cansino]

My module is just a template that loops on an array, where elements
are users with groups belongings and sshkeys.

So, If I want to give access to all machines to sysadmins and
developers the devs ones, my playbook would be:

---
- hosts: all
user: root
vars:
- user_groups:
- sysadmin
roles:
- users
- hosts: dev
user: root
vars:
- user_groups:
- development
roles:
- users

This happens to run twice the same role, one because of all matching,
and the second one the pretended one.

Because of the module using a template, instead of receiving as
parameters sysadmin + development, it receives once sysadmin, and next
development, so it overwrites the first template with the next one.

> You received this message because you are subscribed to a topic in the

> Google Groups "Ansible Project" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/ansible-project/EuLmmfwZmAo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/ansible-project/CAEhzMJBbeQBKLMgxrxVg8mqnjHjTiBia0w7%3Df3miKM4PCcNwcg%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.

--
Javier Domingo Cansino

[Serge van Ginderachter]

OK, I think I see it. There is no such thing as overriding playbooks. Playbooks are a series of plays that are a series of roles and tasks to execute, and tasks use variables.

The overriding here with ansible happens with variables in the inventory.

Given your case, you would define the variable user_groups in the inventory, as a group variable:

group_vars/all 

    - user_groups:
      - sysadmin

​and
group_vars/dev​

 

    - user_groups:
      - development

​And then you only keep the play targetting the all group.​

​Personally, I rarely use vars defined in the playbook, except when I just need some "constant"​.

[Javier Domingo Cansino]

No no, I meant overriding variables. In this case, the variable "user_groups" is an array of the groups that the role "users" has to install in each machine, and it doesn't inherit from all the "sysadmin" group.

I need to know if there is something similar to achieve hierarchies of inheritance for variables. Or a big sample project. I need to understand how stuff is structured, because I find it like too linear atm, not allowing to specify that some machines have different properties makes harder to me to understand the abstraction layers.

I mean not being able to specify exceptions without having them be executed twice. (once for all and another for the specific case).

I understand is because of the knowledge I have and that I am accustomed to puppet, so any example on how to structure a big project would be really helpful. I aim to control everything with ansible, from users, configuration, installed programs, iptables rules, applications deployed, etc.

Thank you,

[Serge van Ginderachter]

​Javier,​

On 19 September 2014 22:36, Javier Domingo Cansino <javi...@gmail.com> wrote:

No no, I meant overriding variables. In this case, the variable "user_groups" is an array of the groups that the role "users" has to install in each machine, and it doesn't inherit from all the "sysadmin" group.

​You are confusing me,

- overriding variables happens within inventory groups, and as documented

- ​you talk about a "sysadmin group". Is this a /etc/group group or an ansible group?

I need to know if there is something similar to achieve hierarchies of inheritance for variables.

​Yes, that's what I was talking about. groups within the inventory have a ​

 hierarch

​y: child groups override​ parent groups.

Check
    http://docs.ansible.com/intro_inventory.html
and the topic of variable precedence
    http://docs.ansible.com/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable

(I'd agree the hierarchy of groups should be better documented though.)

Or a big sample project. I need to understand how stuff is structured, because I find it like too linear atm, not allowing to specify that some machines have different properties makes harder to me to understand the abstraction layers.

​Basically you can define groups of groups, and the deeper the group, the higher the level of importance.​

 

I mean not being able to specify exceptions without having them be executed twice. (once for all and another for the specific case).

​You need to execute on the highest level group​, then modify variables on child and/or grandchild groups.

I understand is because of the knowledge I have and that I am accustomed to puppet, so any example on how to structure a big project would be really helpful. I aim to control everything with ansible, from users, configuration, installed programs, iptables rules, applications deployed, etc.

​I'm actually not aware of extensive examples​

 

​of group hierarchy. Maybe a fellow list member can help here?

  Serge​

[Brian Coca]

I think an example can be clearer, your play would look like this:

- hosts: all
user: root

roles:
- users

now in group_vars/all you have

user_groups:
- sysadmins

and in group_vars/dev you have

user_groups:
- sysadmins
- development

This is using the default strategy that overwrites, if you set it to
merge, you don't need to add the sysadmins in the devs file.

[Michael DeHaan]

On Fri, Sep 19, 2014 at 12:58 PM, Javier Domingo Cansino <javi...@gmail.com> wrote:

Hello,

First of all, I am coming from puppet. I have already done some experiments with ansible, and created a user management module, mainly to create the root Authkeys file. This was a little approach to see how overrides etc are handled by ansible.

The result was having the module executed twice, one for the general case and the other for the specific one. Because of this, I wondered whether it would be possible to have overrides correctly done.

I have read a thread asking for Ansible's hiera, and for what I understood, you proposed using lookups and external inventories. I see ansible has a different way to do stuff, but I don't understand how this sort of features would provide such flexibility.

Yeah, you don't want.

That is a confusing overdesigned hack to patch over something Puppet couldn't do well originally, and something that Ansible was designed for early on.

Ansible provides all the group and inventory management *out of the box*, actually.

group_vars, host_vars, etc, all required without using an external tool.

 

I would be glad if someone could help me understand how to correctly structure Ansible code for my use case.

[Javier Domingo Cansino]

One last questions. I learnt how to add new values (ex. devs can
access dev), but how may I selectively use merge or not strategy?

I mean, I want overrides on user_groups vars, but not in all of them.
In puppet it exists lookup_array, lookup_hash for merges, and
lookup_hiera for overriding.

For example,
--- groups_all
- users_groups:
- sysadmin
--- groups_dev
- users_groups:
- devs

lookup_hiera(users_groups) => ['devs']
lookup_array(users_groups) => ['devs', 'sysadmin']

So that I can choose to locally use some merge configs.

Thanks for the tips btw, I found strange the names but just because of
not being used to =)

> You received this message because you are subscribed to a topic in the

> Google Groups "Ansible Project" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/ansible-project/EuLmmfwZmAo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgxg9XvCS4eEMvKPVDCuMPmYDn42MWSuERG-JoW2B83ckw%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.

--
Javier Domingo Cansino

[Brian Coca]

you could always compose by env:

in group_vars/dev:

users: "{{devs|union(sysadmins)}}

in group_vars/prod:

users: "{{sysadmins}}"



--
Brian Coca
Stultorum infinitus est numerus
0110000101110010011001010110111000100111011101000010000001111001011011110111010100100000011100110110110101100001011100100111010000100001
Pedo mellon a minno

[Javier Domingo Cansino]

The problem with that is that I have to say sysadmin...

wouldn't it be possible to have something like:

in group_vars/all:

groups: sysadmins

in group_vars/dev:

users: "{{devs|union(all)}}

in group_vars/prod:

users: "{{nsoc|union(all}}"

For example?

The idea is to specify "Merge this with the higher level result" or
"Substitute with this".

> --
> You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/EuLmmfwZmAo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHsyMtDAQqtub7jEo_sJviS4BpMn877%2BYnNTWdgiVb9G1GA%40mail.gmail.com.

[Brian Coca]

no, union takes a list, not a inventory file name

you can set global merge behavior but not make it per item, not sure
why specify the file works better than specify the list in the file

[Javier Domingo Cansino]

The idea is that if I want to merge top to bottom, makes no sense if I
have to put all the vars of the top file in the bottom file.

I will try to explore a little bit if this is something I can achieve somehow...

> --
> You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/EuLmmfwZmAo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHsz5yoTkxG16ABPTBP72JiZ18pY0qjwMf-eh%2BEJ1H5g0Ww%40mail.gmail.com.