idea: allow group_vars/ and host_vars/ files to be organised in sub folders

[Serge van Ginderachter]

Hi list,

My use case:

At $WORK, we divise our ansible repo in multiple projects/* dirs, where all projects have their own hosts file and group_vars/; a wrapper script merges all these together in $basedir; basically having all group_vars files in one big directory is too big here (we have 800 hosts and almost 300 groups).

(This should better be handled by some backend + inventory script, but for now, that's what we have. )

Now, one optimisation, would be for us to get rid of that part in the wrapper script, basically allowing to organise goup_vars/ and host_vars/ files into subdirectories.

Basically what I want is, instead of having

group_vars/

group_vars/project1group1

group_vars/project1group2

group_vars/project2group1

group_vars/project2group2

I want to have this:

group_vars/

group_vars/project1/project1group1

group_vars/project1/project1group2

group_vars/project1/project2group1

group_vars/project1/project2group2

The only problem I see with that, is that would allow to have multiple files for the same group, such as

group_vars/

group_vars/project1/project1group1

group_vars/project2/project1group1

Which would probably confusing, and not clear which logic is implemented on which variable gets precedence when that same variable is defined in multiple files with the same group name, hence for the same group.

(in our case, we just wouldn't have that, by convention, but it would be possible to abuse)

group_vars/ and host_vars/ are handled by the vars_plugins/group_vars plugin, and looking at the code, that could be done with a small and simple patch.

Your ideas?

      Serge

[Michael DeHaan]

How would ansible know which subdirectory to use?

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Serge van Ginderachter]

On 12 April 2013 15:53, Michael DeHaan <mic...@ansibleworks.com> wrote:

How would ansible know which subdirectory to use?

​It doesn't have to, just parse all the files it can find.

The intermediatie $project directory is just for organisational purposes.

As you said on irc, parsing all leaf nodes with matching filenames.

  Serge​

[Michael DeHaan]

I'm fine with this, we should make it also ignore file extensions at the same time.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Govinda Fichtner]

Is this already implemented in a current Ansible version?

[Serge van Ginderachter]

On 23 July 2014 18:01, Govinda Fichtner <govinda....@googlemail.com> wrote:

Is this already implemented in a current Ansible version?

​Yes, instead of e.g. a group_vars/groupname.yml, you can​ now have group_vars/groupname/ directory with multiple files in it.

[r.bet...@vehikl.com]

I just tried this on 1.9.1 I have /group_vars/staging/staging.yml and /group_vars/staging/secrets.yml and it seems to me that it is not merging the files properly.

I have an api variable in the staging file which doesnt exist in the secrets file. When I run the playbook it complains that the api variable doesnt exist. If I either remove the secrets file or add the api variable to the secrets file then it works.

[Serge van Ginderachter]

On 7 August 2015 at 20:17, <r.bet...@vehikl.com> wrote:

I just tried this on 1.9.1 I have /group_vars/staging/staging.yml and /group_vars/staging/secrets.yml and it seems to me that it is not merging the files properly.

I have an api variable in the staging file which doesnt exist in the secrets file. When I run the playbook it complains that the api variable doesnt exist. If I either remove the secrets file or add the api variable to the secrets file then it works.

​If that would be true it would be a bug, and you would need to present us a minimal example on how to reproduce it.

That being said, I use ​this feature everywhere around and i know i works. It might be some corner case of course, which would only become clear when being able to reproduce it.

Can you make a minimal example that reproduces this, and show us the needed files?

Thanks,

Serge

[Sean Cavanaugh]

I am curious if there was follow-up to this.  I am seeing something similar for a customer I am working on.

• If I put all my variables under group_vars/all (where all is a flat file in yml syntax) it works fine.
  
• If I move one device out of group_vars/all to group_vars/leaf/leaf01 all devices work great.  (where leaf corresponds to a group within my Ansible hosts file).
• As soon as I do group_vars/leaf/leaf01 and leaf02 it can't find leaf01 anymore.  leaf02 will work but not leaf01.  (where leaf01 and leaf02 are hosts in my ansible hosts file both in the group leaf)

Do they have to end with .yml?  Is this not how group_vars works?  Just trying to make folders to break up variables for simplification. 

(I am running on Ansible 2.1.1)

[Dick Davies]

Are you sure that's the way things are supposed to work?

I'd have expected individual hosts to be under host_vars/leaf01 , not
nested under a group (remember, a host can be in multiple groups).

so your layout would be more:

.
├── group_vars
│ ├── all
│ └── leaf
├── host_vars
│ ├── leaf01
│ └── leaf02
└── hosts


On 10 September 2016 at 19:02, 'Sean Cavanaugh' via Ansible Project

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.

> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/95184094-6d7f-41f7-94aa-8d16521025d8%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Sean Cavanaugh]

In this thread though Dehaan said this should work->

I just tried this on 1.9.1 I have /group_vars/staging/staging.yml and /group_vars/staging/secrets.yml and it seems to me that it is not merging the files properly.

But others said this was a problem.  I just don't think this is documented well.  If its not supposed to work I guess it shouldn't work :(

[Kai Stian Olstad]

On 12.09.2016 15:15, 'Sean Cavanaugh' via Ansible Project wrote:
> In this thread though Dehaan said this should work->
>
>> I just tried this on 1.9.1 I have /group_vars/staging/staging.yml and
>> /group_vars/staging/secrets.yml and it seems to me that it is not
>> merging
>> the files properly.
>>
>
> But others said this was a problem. I just don't think this is
> documented
> well. If its not supposed to work I guess it shouldn't work :(

According to the documentation it should work
https://docs.ansible.com/ansible/intro_inventory.html#splitting-out-host-and-group-specific-data

Quote from the page:
"As an advanced use-case, you can create directories named after your
groups or hosts, and Ansible will read all the files in these
directories. An example with the ‘raleigh’ group:

/etc/ansible/group_vars/raleigh/db_settings
/etc/ansible/group_vars/raleigh/cluster_settings

All hosts that are in the ‘raleigh’ group will have the variables
defined in these files available to them. This can be very useful to
keep your variables organized when a single file starts to be too big,
or when you want to use Ansible Vault on a part of a group’s variables.
Note that this only works on Ansible 1.4 or later."

--
Kai Stian Olstad

[Andrea Dell'Amico]

> On 14 Sep 2016, at 08:04, Kai Stian Olstad <ansible-pr...@olstad.com> wrote:
>
> On 12.09.2016 15:15, 'Sean Cavanaugh' via Ansible Project wrote:
>> In this thread though Dehaan said this should work->
>>> I just tried this on 1.9.1 I have /group_vars/staging/staging.yml and
>>> /group_vars/staging/secrets.yml and it seems to me that it is not merging
>>> the files properly.
>> But others said this was a problem. I just don't think this is documented
>> well. If its not supposed to work I guess it shouldn't work :(
>
> According to the documentation it should work
> https://docs.ansible.com/ansible/intro_inventory.html#splitting-out-host-and-group-specific-data

And it does. I use this feature heavily since before ansible 1.9, never had a problem.
Be careful to not duplicate variables definitions. Starting from version 2 ansible issue a warning if a variable has been defined more than once, but if I remember well it was silent on older versions.

> --
> Kai Stian Olstad

Ciao
Andrea

--
Andrea Dell'Amico
http://adellam.sevenseas.org/