Inventory-specific group_vars/all overrides

[Trevor Hartman]

Hi, I don't see my issue addressed in the docs or existing issues. I'm trying to override site-wide group_var/all config with an inventory-specific "all" but it looks like the site-wide vars get precedence, unexpectedly. Here's my structure:

env

├── dev

│   ├── group_vars

│   │   ├── all

│   │   └── node

│   └── inventory.py

└── production

    ├── group_vars

    │   └── all

    └── inventory.py

group_vars

└── all

I have a simple debug playbook to test vars: ansible-playbook -i env/dev debug.yml

1. In group_vars/all, I set env: common
2. In env/dev/group_vars/all I set env: common/dev, but it does not override the "common" value (unless I remove env: common from group_vars/all
   
3. In env/dev/group_vars/node I set env: common/dev/node, and it does override the "common" value for any hosts that belong to the "node" group.

In #2, shouldn't env/dev/group_vars/all be overriding the value set from group_vars/all?

Thanks,

Trevor

[Brian Coca]

No, the play specific group_vars is there for overriding inventory
group_vars, this works as designed.

In general inventory is something you normally share across projects,
plays are more specific to the action you want to accomplish, this is
the logic followed for the above behavior.
--
Brian Coca

[Trevor Hartman]

Forgot to specify ansible version: ansible 1.8.2

[Trevor Hartman]

Sorry, not following you. In this example I'm not storing any vars in playbooks.

I find it strange that group-specific vars at env/dev/group_vars/node override group_vars/all, but env/dev/group_vars/all does not.

It sounds like that's expected, so it's not a bug, but I don't understand the logic behind it.

An example of a var I'd like defined as a default but may want to override on a per-environment basis is version. Let's say I want my default to be 0.9.0, but in production I want to override the default and specify 0.8.0.

[Brian Coca]

Your plays are at the level of top env (assumed from how you call
debug.yml) and they load that group_vars, this is how you load
inventory vars at the play level (different from play vars). It is
meant to override the inventory as plays are more specific than
inventory.

if you do not want this to be the case, put the inventory outside the same hier,

env/
plays/

Then it would be ansible-playbook -i ../env/dev debug.yml and the
../env/group_vars/all will NOT be sourced.
You are mixing 2 loading methods, one is part of inventory, the other
part of plays, I think this is what is confusing you.

--
Brian Coca

[Trevor Hartman]

Thanks for explaining that.

It seem I have a pretty common use case, so it feels strange there isn't a more direct way of solving it. I see what you mean by mixing 2 loading methods though. I structured it this way following http://toja.io/using-host-and-group-vars-files-in-ansible/

There were other blog posts about multistage ansible environments, but this one was by far the simplest.

As a workaround, I might have my dynamic inventory script put all groups in a single supergroup "all_groups" to allow me to override vars at the env-level (e.g. env/dev/group_vars/all_groups). Or is there a better way to specify a var default at the inventory level (instead of play level)?

[Brian Coca]

i think the example you posted/followed is a bit confusing because
their play is named group_vars.yml, it does not have an actual
group_vars dir at play level.



--
Brian Coca

[Trevor Hartman]

You're right, adding group_vars/all was my own attempt at providing site-wide defaults that could be overriden at the env level.

So it sounds like I need a way to provide site-wide defaults at the Inventory level instead of Play level. I have an idea to manually load a defaults file with my dynamic inventory script and setting those vars, but it's a bit ugly.

[Azul Inho]

I use a different approach,

root

├── inventory

│   |-- dev

│   │-- vagrant

│   │-- preprod

│   └── production
|

└── group_vars
    |
    |-- all.yaml
    |-- dev/
    |    |-- secrets.yaml
    |
    |-- vagrant/
    |    |-- secrets.yaml
    |    
    |-- preprod/
    |    |-- secrets.yaml
    |
    |-- production/
         |-- secrets.yaml

where my group_vars/all.yaml contains all my definitions for every environment, I keep it DRY by using lots of yaml anchors/aliases.
https://github.com/Azulinho/ansible-jenkins-showcase/blob/master/group_vars/all.yaml#L686

And in my inventory files, I set a variable 'deploy_env'
https://github.com/Azulinho/ansible-jenkins-showcase/blob/master/vagrant

I consume that 'deploy_env' variable (typically matches my inventory file) at the end of my group_vars/all.yaml file.
https://github.com/Azulinho/ansible-jenkins-showcase/blob/master/group_vars/all.yaml#L713

there are a gazillion ways of doing it, this one works well for me.

   

[Trevor Hartman]

Thanks for posting Azul. It's interesting to see another way of handling it. I ended up with this structure, which I'm liking a lot:

env

├── defaults

├── dev

│   ├── group_vars

│   │   ├── backend

│   │   ├── defaults

│   │   ├── ingest

│   │   ├── monitoring

│   │   └── zookeeper

│   └── inventory.py

└── production

    ├── group_vars

    │   ├── backend

    │   ├── defaults

    │   ├── ingest

    │   ├── monitoring

    │   └── zookeeper

    └── inventory.py

1. I completely eschew Ansible's "all" supergroup because precedence was not behaving as I expected.

2. I use a dynamic inventory script which adds all my groups to a "defaults" supergroup. This is my "all" replacement, so env/defaults is my new group_vars/all. The script also manually reads in env/defaults and sets the vars in the JSON output.

3. env/defaults can specify site-wide defaults which can be overridden by env/dev/group_vars/defaults. There's a third level of specificity at env/dev/group_vars/$group, e.g. env/dev/group_vars/zookeeper, which would override anything in env/dev/group_vars/defaults.

I'd really like to see Ansible create an official solution to this common problem. It'd take the burden off devs and increase uniformity in Ansible automation.

One awkward piece of my setup is having to set all my vars at the inventory level, when much of it belongs at the playbook level.

[James Morgan]

Hi,

I am seeing a similar problem with my inventory variables. I have the following defined

./group_vars/all.yml: yum_epel_ansible_managed: false

./inventory/db-builder/group_vars/all.yml: yum_epel_ansible_managed: true

Prior to 1.8.1 when running the db-builder inventory this would be overridden as true

Using 1.8.1 this is now left as false

"yum_epel_ansible_managed": "false"

We haven't changed our playbooks or inventories. 

This is causing us issues as you'd imagine. Is this approach incorrect as it has worked fine previously

Many thanks 

James

[James Morgan]

Noticed this https://github.com/ansible/ansible/issues/9877 though its a little unclear what the approach should be.

In my example I can make variable a role default which would fix it though I need to go and change every instance of this precedence issue.

The old behaviour seems logical

[Patrick Martini]

Hi Trevor,

can you share your dynamic inventory script with us ? it looks like the only way to have site-wide defaults.

Thanks 

Patrick 

[Trevor Hartman]

Hi Patrick,

I'm using the https://github.com/lukaspustina/dynamic-inventory-for-ansible-with-openstack dynamic inventory script, specifically for obtaining inventory from C3. You can use it as a reference if you like.

I'm still using the directory structure I posted above, with group_vars inside each env. The "defaults" group is the special one that I use to replace Ansible's normal "all" group, except that I get expected override precedence. I modified the dynamic inventory script to load the non-env-specific defaults file and apply them like this:

  def read_default_vars(self):

    c = ConfigParser.ConfigParser()

    c.read("env/defaults")

    section = "all"

    result = {}

    for key in c.options(section):

      result[key] = c.get(section, key)

    return result

  def build_defaults_parent_group(self):

    groups = list(self.inventory().iterkeys())

    groups.remove("_meta")

    default_vars = self.read_default_vars()

    self.inventory()['defaults'] = {

        'hosts': [],

        'children': groups,

        'vars': default_vars}

 

That way the env-specific defaults file (e.g. env/dev/group_vars/defaults) will override any vars set at env/defaults.

It is still awkward. I wish Ansible had a built-in solution.