two bugs in the group depth parsing logic yielding wrong and inconsistently wrong depth values for groups

[Serge van Ginderachter]

Hi,

As discussed with Michael on irc yesterday, I suspected something to be wrong with the calculated group depth, which is AFAICS used to have the right variable precedence according to the depth of a group in the parent-child tree.

As I'm looking into making a custom inventory script, tailored to our very strict standardized environment, I wanted to have a way to more or less visualize the group tree, and be able to check it on possible errors.

To test this (and also to troubleshoot other inventory related stuff) I wrote an "ansible-vars" script based on bin/ansible that parses the inventory, and can show a (tree) list of groups, and a list of hosts with all of their variables. It's not super clean, but gets the job done for now. (If interested, you can check it out here: https://github.com/ansible-contrib/ansible-plugins/blob/devel/bin/ansible-vars )

It took me some time to figure this out, as the wrong depth values I got were inconsistent

- depended on the level of subdirectories in which the ini files resided

- depended on the order of groups as noted in the ini files

As it turned out, those were two similar, but different bugs in group.y and dir.py

Example:

######################################

$ cat inventory/ini/hosts

[tomcat:children]

depth3b

[depth1:children]

depth2

[depth2:children]

depth3a

depth3b

[depth3a]

node1

node2

[depth3b]

node3

node4

[appservers:children]

depth3a

$ cat inventory/ini/hosts2 

[emil:children]

emil-on

emil-oe

emil-pr

[emil-on]

emil-on-1-mgt

emil-on-2-mgt

[emil-oe]

emil-oe-1-mgt

emil-oe-2-mgt

[emil-pr]

emil-pr-1-mgt

emil-pr-2-mgt

[tomcat:children]

emil

$ ansible-vars --list-groups -i inventory/ini/hosts

 all(0) | p:  | c:ungrouped(1),tomcat(1),depth1(1),depth2(2),depth3a(3),depth3b(3),appservers(1)

. appservers(1) | p: all(0) | c:depth3a(3)

. depth1(1) | p: all(0) | c:depth2(2)

.. depth2(2) | p: all(0),depth1(1) | c:depth3a(3),depth3b(3)

... depth3a(3) | p: all(0),depth2(2),appservers(1) | c:

... depth3b(3) | p: all(0),tomcat(1),depth2(2) | c:

. tomcat(1) | p: all(0) | c:depth3b(3)

. ungrouped(1) | p: all(0) | c:

$ ansible-vars --list-groups -i inventory/ini/hosts2 

 all(0) | p:  | c:ungrouped(1),emil(2),emil-on(2),emil-oe(2),emil-pr(2),tomcat(1)

. tomcat(1) | p: all(0) | c:emil(2)

.. emil(2) | p: all(0),tomcat(1) | c:emil-on(2),emil-oe(2),emil-pr(2)

. ungrouped(1) | p: all(0) | c:

######################################

Note: the output of my script is a bit redundant: it starts printing every group with (depth) and list of child (c) and parent (p) groups, then printing every child group beneath it, indented, *only if* it's a direct child (depth+1)

The first ini files by itself turns out to have correct results, but as you can see, all the emil groups have all the same depth, whilst emil is a parent of emil-*

This can be fixed by moving the tomcat group at the top.

For the depth logic to work as of now, you need to put top level groups first, etc.

Combining these to ini files by pointing the inventory to the ini dir, gives us other results:

######################################

$ ansible-vars --list-groups -i inventory/ini

 all(0) | p:  | c:depth3a(1),tomcat(1),depth3b(2),ungrouped(1),depth2(1),depth1(1),appservers(1),emil(2),emil-on(3),emil-oe(3),ungrouped(1),tomcat(1),emil-pr(3)

. appservers(1) | p: all(0) | c:depth3a(1)

. depth1(1) | p: all(0) | c:depth3a(1),depth3b(2),depth2(1)

.. depth3b(2) | p: depth2(1),all(0),depth1(1),tomcat(1) | c:

. depth2(1) | p: all(0),depth1(1) | c:depth3a(1),depth3b(2)

.. depth3b(2) | p: depth2(1),all(0),depth1(1),tomcat(1) | c:

. depth3a(1) | p: depth2(1),all(0),depth1(1),appservers(1) | c:

. tomcat(1) | p: all(0),all(0) | c:depth3b(2),emil(2),emil-on(3),emil-oe(3),emil-pr(3)

.. depth3b(2) | p: depth2(1),all(0),depth1(1),tomcat(1) | c:

.. emil(2) | p: all(0),tomcat(1) | c:emil-on(3),emil-oe(3),emil-pr(3)

... emil-oe(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-on(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-pr(3) | p: all(0),emil(2),tomcat(1) | c:

. tomcat(1) | p: all(0),all(0) | c:depth3b(2),emil(2),emil-on(3),emil-oe(3),emil-pr(3)

.. depth3b(2) | p: depth2(1),all(0),depth1(1),tomcat(1) | c:

.. emil(2) | p: all(0),tomcat(1) | c:emil-on(3),emil-oe(3),emil-pr(3)

... emil-oe(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-on(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-pr(3) | p: all(0),emil(2),tomcat(1) | c:

. ungrouped(1) | p: all(0),all(0) | c:

. ungrouped(1) | p: all(0),all(0) | c:

######################################

And again different when using the parent dir of that dir:

######################################

$ ansible-vars --list-groups -i inventory

 all(0) | p:  | c:depth3a(1),emil(1),tomcat(1),depth3b(2),emil-oe(2),ungrouped(1),depth2(1),depth1(1),emil-on(2),emil-pr(2),appservers(1)

. appservers(1) | p: all(0) | c:depth3a(1)

. depth1(1) | p: all(0) | c:depth3a(1),depth3b(2),depth2(1)

.. depth3b(2) | p: depth2(1),all(0),depth1(1),tomcat(1) | c:

. depth2(1) | p: all(0),depth1(1) | c:depth3a(1),depth3b(2)

.. depth3b(2) | p: depth2(1),all(0),depth1(1),tomcat(1) | c:

. depth3a(1) | p: depth2(1),all(0),depth1(1),appservers(1) | c:

. emil(1) | p: all(0),tomcat(1) | c:emil-oe(2),emil-on(2),emil-pr(2)

.. emil-oe(2) | p: all(0),emil(1),tomcat(1) | c:

.. emil-on(2) | p: all(0),emil(1),tomcat(1) | c:

.. emil-pr(2) | p: all(0),emil(1),tomcat(1) | c:

. tomcat(1) | p: all(0) | c:emil(1),depth3b(2),emil-oe(2),emil-on(2),emil-pr(2)

.. depth3b(2) | p: depth2(1),all(0),depth1(1),tomcat(1) | c:

.. emil-oe(2) | p: all(0),emil(1),tomcat(1) | c:

.. emil-on(2) | p: all(0),emil(1),tomcat(1) | c:

.. emil-pr(2) | p: all(0),emil(1),tomcat(1) | c:

. ungrouped(1) | p: all(0) | c:

######################################

Note that this last run actually cleaned up some groups that got defined multiple times.

I managed to write two patches that seem to solve this:

######################################

$ ansible-vars --list-groups -i inventory | grep -v debug

 all(0) | p:  | c:tomcat(1),ungrouped(1),appservers(1),depth1(1),emil(2),depth2(2),depth3a(3),depth3b(3),emil-oe(3),emil-on(3),emil-pr(3)

. appservers(1) | p: all(0) | c:depth3a(3)

. depth1(1) | p: all(0) | c:depth2(2),depth3a(3),depth3b(3)

.. depth2(2) | p: all(0),depth1(1) | c:depth3a(3),depth3b(3)

... depth3a(3) | p: depth2(2),all(0),depth1(1),appservers(1) | c:

... depth3b(3) | p: depth2(2),all(0),depth1(1),tomcat(1) | c:

. tomcat(1) | p: all(0) | c:emil(2),depth3b(3),emil-oe(3),emil-on(3),emil-pr(3)

.. emil(2) | p: all(0),tomcat(1) | c:emil-oe(3),emil-on(3),emil-pr(3)

... emil-oe(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-on(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-pr(3) | p: all(0),emil(2),tomcat(1) | c:

. ungrouped(1) | p: all(0) | c:

$ ansible-vars --list-groups -i inventory/ini/ | grep -v debug

 all(0) | p:  | c:tomcat(1),ungrouped(1),depth1(1),appservers(1),depth2(2),depth3a(3),depth3b(3),ungrouped(1),tomcat(1),emil(2),emil-on(3),emil-oe(3),emil-pr(3)

. appservers(1) | p: all(0) | c:depth3a(3)

. depth1(1) | p: all(0) | c:depth2(2),depth3a(3),depth3b(3)

.. depth2(2) | p: all(0),depth1(1) | c:depth3a(3),depth3b(3)

... depth3a(3) | p: depth2(2),all(0),depth1(1),appservers(1) | c:

... depth3b(3) | p: depth2(2),all(0),depth1(1),tomcat(1) | c:

. tomcat(1) | p: all(0),all(0) | c:depth3b(3),emil(2),emil-on(3),emil-oe(3),emil-pr(3)

.. emil(2) | p: all(0),tomcat(1) | c:emil-on(3),emil-oe(3),emil-pr(3)

... emil-oe(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-on(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-pr(3) | p: all(0),emil(2),tomcat(1) | c:

. tomcat(1) | p: all(0),all(0) | c:depth3b(3),emil(2),emil-on(3),emil-oe(3),emil-pr(3)

.. emil(2) | p: all(0),tomcat(1) | c:emil-on(3),emil-oe(3),emil-pr(3)

... emil-oe(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-on(3) | p: all(0),emil(2),tomcat(1) | c:

... emil-pr(3) | p: all(0),emil(2),tomcat(1) | c:

. ungrouped(1) | p: all(0),all(0) | c:

. ungrouped(1) | p: all(0),all(0) | c:

$ ansible-vars --list-groups -i inventory/ini/hosts | grep -v debug

 all(0) | p:  | c:ungrouped(1),tomcat(1),depth1(1),depth2(2),depth3a(3),depth3b(3),appservers(1)

. appservers(1) | p: all(0) | c:depth3a(3)

. depth1(1) | p: all(0) | c:depth2(2)

.. depth2(2) | p: all(0),depth1(1) | c:depth3a(3),depth3b(3)

... depth3a(3) | p: all(0),depth2(2),appservers(1) | c:

... depth3b(3) | p: all(0),tomcat(1),depth2(2) | c:

. tomcat(1) | p: all(0) | c:depth3b(3)

. ungrouped(1) | p: all(0) | c:

$ ansible-vars --list-groups -i inventory/ini/hosts2 | grep -v debug

 all(0) | p:  | c:ungrouped(1),emil(2),emil-on(3),emil-oe(3),emil-pr(3),tomcat(1)

. tomcat(1) | p: all(0) | c:emil(2)

.. emil(2) | p: all(0),tomcat(1) | c:emil-on(3),emil-oe(3),emil-pr(3)

... emil-oe(3) | p: all(0),emil(2) | c:

... emil-on(3) | p: all(0),emil(2) | c:

... emil-pr(3) | p: all(0),emil(2) | c:

. ungrouped(1) | p: all(0) | c:

######################################

The duplicaton of groups mentioned in multiple ini files is not solved here yet. I don't think that's a bug in my script, but this still could be possible.

Either way, this is less of a problem, as that doesn't seem to affect ansible, the host list is still generated correctly:

######################################

$ ansible-vars --list-hosts -i inventory | grep -v debug

emil-oe-1-mgt

emil-oe-2-mgt

emil-on-1-mgt

emil-on-2-mgt

emil-pr-1-mgt

emil-pr-2-mgt

node1

node2

node3

node4

[grpdpthtstng] serge@goldorak:~/src/ansible$ ansible-vars --list-hosts -i inventory/ini/ | grep -v debug

emil-oe-1-mgt

emil-oe-2-mgt

emil-on-1-mgt

emil-on-2-mgt

emil-pr-1-mgt

emil-pr-2-mgt

node1

node2

node3

node4

######################################

I made a test branch on my fork containing my ansible-vars script and the test inventory, so anyone can reproduce my findings:

https://github.com/sergevanginderachter/ansible/tree/groupdepthtesting

Please note: 

- at HEAD~1 you have the latest ansible + script + testinventory

- at HEAD you also have my patches containing some print 'debug' statements, which help to see what's going on at parsing time.

Whilst this is actually quite a fundamental bug, I think, I suspect not many people will have bumped into this, as I don't think many people will nest group variables and using precedence at many levels.

I hope this all makes it clear enough :)

Thanks for feedback,

     Serge

[Michael DeHaan]

I'm not sure the internal variables in Ansible being weird for you are a symptom.   What's actually the problem?

As nicely as possible, please keep "TLDR" in mind a bit :)

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Serge van Ginderachter]

On 17 May 2013 16:41, Michael DeHaan <mic...@ansibleworks.com> wrote:

I'm not sure the internal variables in Ansible being weird for you are a symptom.   What's actually the problem?

As I said earlier, the effect on variable precedence can be a corner case. I started assembling a test inventory with some variables, and managed to find a setup where the problem shows.

Inventory file, used commands, the expected result and the (wrong) command output can be found in this gist:

https://gist.github.com/sergevanginderachter/5605645

one example:

hosts file contains groups with this structure; all the app* groups have a variable set to the name of the group

. site

.. apache

... app2

.... app2-dev --> these level groups contain the nodes

.... app2-prod

.. tomcat

... app1

.... app1-dev

.... app1-prod

running a debug task showing that var:

app1-dev-node | success >> {"msg": "var=app1"}

app1-prod-node | success >> {"msg": "var=app1-prod"}

app2-dev-node | success >> {"msg": "var=app2-dev"}

app2-prod-node | success >> {"msg": "var=app2"}

So some nodes get their var from app1/2 instead of the subgroup with the environment.

- pointing the inventory to the hosts file directly, or the parent dir also yields different results

- restructuring the order in the ini hosts file can also influence this

- in the mean time I discovered that the patch I made does not allways solve the problem

More details in the gist.

    Serge

[Michael DeHaan]

Please forgive my lack of brain processing power.

So if I understand this correctly there are variables defined in the tomcat subtree and variables in the apache subtree, and you'd like them to override in some way?   Or which two groups are you not seeing variables blended like you like from?

This is why I strongly encourage the use of group_vars/<groupname> to put variables in, makes inventory much easier to read.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Serge van Ginderachter]

On 19 May 2013 01:59, Michael DeHaan <mic...@ansibleworks.com> wrote:

So if I understand this correctly there are variables defined in the tomcat subtree and variables in the apache subtree, and you'd like them to override in some way?   Or which two groups are you not seeing variables blended like you like from?

To keep things simple, just look that part of my example where the variable 'var' is defined on the app1 and app2 group, and on their respective subgroups. The four '-node' hosts are only defined in those subgroups, and only on 1 single place​​, so there is no overlap, except for parent and child groups 'app*'

Basically, I expect to be able to define var on the app1/2 groups, as a default, and to override them in the childgroups app1/2-dev/prod (in which the nodes are directly defined, and have the greatest depth.

This is how precedence is documented: "" If a variable is defined in multiple groups and one group is a child of the other, the child group variable will override the variable set in the parent. ""

My tests show that this doesn't allways happen.

This is why I strongly encourage the use of group_vars/<groupname> to put variables in, makes inventory much easier to read.

I normally don't use variables defined in ini files, just did that now now for the sake of giving a complete inventory file.​ to test.

 ​Serge​

[Michael DeHaan]

"To keep things simple, just look that part of my example "

I'm sorry, your example is not simple.   

This is why it is very important to submit *minimal* examples and work things down to the smallest possible question.

Just define as few hosts and groups and variables as you possibly can to show this, and pastebin a gist of that.

If it can be done with one variable and three groups, show me that.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Serge van Ginderachter]

On 19 May 2013 15:39, Michael DeHaan <mic...@ansibleworks.com> wrote:

I'm sorry, your example is not simple.

The problem isn't simple also.

​​

This is why it is very important to submit *minimal* examples and work things down to the smallest possible question.

Just define as few hosts and groups and variables as you possibly can to show this, and pastebin a gist of that.

I already tried that, it's not easy to find good examples of showing a problem that only affects in certain conditions, in more complex environments. ​​

If it can be done with one variable and three groups, show me that.

​My example does almost that, it only needs some other groups to trigger the problem/

I might find an example with one variable, defined on two or three group levels, and having some extra groups where no variable is defined. I'll look into that tonight or tomorrow.

  Serge

​

[Michael DeHaan]

Maybe you can draw up some ascii art that would help me understand better, something like this:

http://pastebin.com/PYm47hcu

?

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Michael DeHaan]

Sorry, corrected that a bit, this should be more clear for what I am looking for trying to understand:

http://pastebin.com/vzUrW6Ha

[Michael DeHaan]

Rarrgh :)

Use this one - http://pastebin.com/vCRajgjX

[Serge van Ginderachter]

​I managed to dumb it further down. This example doesn't show all the possible effects when using the 'hosts' ini file directly ​as the inventory, but it does show when putting the hosts file in a subdirectory

Keep in mind though that the problem can show up in more complex examples, when only using an ini file as inventory. I should be able to make a more complex example to show that.

So I made a simple ini file, also with a small graph of the group tree, containing 2 levels of groups, 2 parent groups with each 2 child groups, 1 node per child group, 4 nodes in total.

One single variable 'var' is defined on each of the groups. Each node is expected to get the variable of the respective child group it is the sole member of. 

​O​

utput shows that

​some​

 nodes get the variable from the parent group

​, in this case when pointing the inventory to a dir containing the ini file.​

​

U

​pdated gist: https://gist.github.com/sergevanginderachter/5605645​

​

I hope this will be what you expect :-)

   Serge

[Serge van Ginderachter]

Michael, 

Did you have the time to look into this yet?

Do you need more information?

Serge

[Michael DeHaan]

Haven't had any time yet, no.

Please copy this into a github ticket so we don't lose it.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Serge van Ginderachter]

On 23 May 2013 23:00, Michael DeHaan <mic...@ansibleworks.com> wrote:

Please copy this into a github ticket so we don't lose it.

​https://github.com/ansible/ansible/issues/3009​