variables messed up when using including tasks

[silverdr]

Hello, group, I encountered a strange problem with variables when running a role. The role has some defaults and the variables are used throughout the plays. At some point in the tasks/mains playbook, there are included other tasks from the same directory. Those also include tasks from root directory. Everything seems fine but when the plays AFTER the included are executed then the variables are no longer valid (seems like they are taken from another role's). None of the included tasks changes the variables in question. Seems like a bug? Or am I missing something.

[silverdr]

Actually the problem shows up even when the included tasks do not include any other ones.

[Brian Coca]

I'm not understanding the problem, can you post an acual example and also what versions you are using?

[silv...@srebrnysen.com]

> On 2015-12-09, at 00:10, Brian Coca <bc...@ansible.com> wrote:
>
> I'm not understanding the problem, can you post an acual example and also what versions you are using?

There are several roles with practically the same structures:

- role0
- defaults
- main.yml
- tasks
- main.yml
- task1.yml
- task2.yml
- etc.yml

- role1
- defaults
- main.yml
- tasks
- main.yml
- task1.yml
- task2.yml
- etc.yml

- role2
- defaults
- main.yml
- tasks
- main.yml
- task1.yml
- task2.yml
- etc.yml

and so on. There are other directories but "vars" are empty in all of them. Each role's "main" task uses variables defined in respective "defaults" but also includes other tasks from the same role and tasks directory. While default values are all fine as long as they are used before the first

- include?: task1.yml

they get their values overwritten with some defaults from another role (sic!) once finished processing the included task file. Now that looks to me like a bug but I might be missing something.

A real example:

# Prerequisities / system dependencies
- name: install python http library
apt: name={{item}} state=present
with_items:
- python-httplib2

# Configuring PHP
- include: php.yml

# Configuring database
- include: database.yml

# Configuring network
- include: network.yml

# Configuring httpd
- include: httpd.yml

# Getting and preparing / installing the code
- name: clone repository
git: repo={{repository}} dest={{destination}} accept_hostkey=yes

When I place git above the included tasks then {{destination}} is correct. When I place it below any of the included tasks, then the destination gets wrong value of a default from a completely different role...


>
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/gWN-uuj7NRI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAJ5XC8mnDeE7uv5M7gypoGjHsrX157KL-qvkn3n5t7G8UyYrJQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Igor Cicimov]

Do you by any chance have some of those vars also defined in the group_vars/all file? In that case they overide the ones in the roles default file causing unexpected results for people unaware of this fact.

[patrycjus...@srebrnysen.com]

> On 2015-12-09, at 22:35, Igor Cicimov <ig...@encompasscorporation.com> wrote:
>
> Do you by any chance have some of those vars also defined in the group_vars/all file? In that case they overide the ones in the roles default file causing unexpected results for people unaware of this fact.

No, that would actually explain - Definitely there is no "destination" for example in any group_vars/, host_vars/, etc. In group_vars/all, there is only a vaulted set of "sensitive" variables like passwords, keys, etc.

[Igor Cicimov]

Strange, because I'm sure every time I call a tasks file from another role I have to explicitly include that role's defaults|vars file via vars_files to be able to use its variables. So you are saying you are seeing this happen without this linking. Do you mind showing us the whole main playbook?

[silv...@srebrnysen.com]

> On 2015-12-10, at 23:13, Igor Cicimov <ig...@encompasscorporation.com> wrote:
>
> Strange, because I'm sure every time I call a tasks file from another role I have to explicitly include that role's defaults|vars file via vars_files to be able to use its variables. So you are saying you are seeing this happen without this linking. Do you mind showing us the whole main playbook?

FYI: I - kind of - worked the problem around by moving the variables from "defaults" to "vars" but this causes other inconveniences when I actually want to overwrite them.

Example playbook ({{destination}} value changes after the first -include:

---
- hosts: api
remote_user: teamcity
sudo: yes
gather_facts: no

pre_tasks:

roles:
- { role: "api" }

tasks:

post_tasks:

************************************
roles/api/tasks/main.yml:

# Prerequisities / system dependencies

- name: install dependencies

apt: name={{item}} state=present
with_items:
- python-httplib2

- supervisor

# Configuring PHP
- include: php.yml

# Getting and preparing / installing the code

- name: if exists - remove previously cloned repository
file:
path: "{{destination}}"
state: absent
when: clear_destination is defined
tags:
- api

- name: clone repository
git:

repo: "{{repository}}"
dest: "{{destination}}"
accept_hostkey: yes
tags:
- api_debug

- name: copy default env file # TODO - process the env file out of env.dist
template: src=env.ini.j2 dest={{destination}}/api/.env
tags: env_file

- name: install using composer
composer: command=install working_dir={{destination}}/api/ no_dev=no

- name: change ownership of the cloned repository
file: dest={{destination}} owner={{owneruser}} group={{ownergroup}} state=directory recurse=yes

# Configuring database
- name: stop supervisor service
supervisorctl: name='api:' state=stopped
tags: api_database

- include: database.yml

- name: run migrations
sudo_user: "{{owneruser}}"
command: php artisan migrate chdir={{destination}}/api/

- name: seed the database
sudo_user: "{{owneruser}}"
command: php artisan db:seed chdir={{destination}}/api/

# Cleaning elastic index
- name: cleanup elastic index
uri: url=http://{{elastic_hostname}}:{{elastic_port}}/{{listing_index}} method=DELETE status_code=200,404
when: clean_elastic_index is defined
tags:
- clean_elastic_index

# Run artisan queue daemon with supervisord
- name: copy queue daemon configuration
template: src=supervisord.conf.j2 dest=/etc/supervisor/conf.d/api.conf
tags: queue

- name: add new service
supervisorctl: name='api:' state=started
tags: queue

- name: start added service
supervisorctl: name='api:' state=restarted
tags: queue

# Configuring network
- include: network.yml

# Configuring httpd
- include: httpd.yml

# Generating apidocs
- include: apidocs.yml

******************************************
roles/api/tasks/php.yml:

---
- name: install php5 extensions

apt: name={{item}} state=present
with_items:

- php5-mcrypt
- php5-json
- php5-pgsql
- php5-xsl
- php5-gmp

- name: enable php5-mcrypt extension
command: php5enmod mcrypt
args:
creates: /etc/php5/apache2/conf.d/20-mcrypt.ini

# when installing xdebug make sure that xdebug.max_nesting_level = 250
# is also set in the php's php.ini configuration file
- name: install php5 xdebug extension
apt: name={{item}} state=present
- php5-xdebug
when: xdebug is defined

- name: set php5 xdebug configuration param
lineinfile:
dest: "{{php_ini_path}}"
backup: true
backrefs: true
state: present
regexp: "{{item.regexp}}"
line: "{{item.line}}"
with_items:
- {regexp: '^xdebug.max_nesting_level', line: 'xdebug.max_nesting_level = 250'}
notify: restart httpd
when: xdebug is defined

[silv...@srebrnysen.com]

> On 2015-12-10, at 23:13, Igor Cicimov <ig...@encompasscorporation.com> wrote:
>

> Strange, because I'm sure every time I call a tasks file from another role I have to explicitly include that role's defaults|vars file via vars_files to be able to use its variables.

Actually I stay within one role. The problem is that when I include other tasks from the same role, the role's defaults change into values from another role, which defaults have variables named the same as in the current one.

[Igor Cicimov]

Just out of curiosity, what happens when you supply destination via extra_vars and leave everything as it was in defaults file?

[Igor Cicimov]

Also if you can put

- debug: var=destination

before and after every include call would be very helpful to see where the var actually gets changed.

Just realized that my previous message might not be clear enough, what I mean is add:

--extra-vars '{"destination":"some_value_here"}'

to the ansible-playbook command and move back all your vars in their respective defaults files as they were before.

[silv...@srebrnysen.com]

I /think/ I know where the problem comes from: I encountered it with group vars defined in the inventory file. I eventually moved both the role variables and group_vars into inventory file, trying to establish one place where the variables are defined. Now:

My inventory file looks like this (IPs are dummy):

[group0]
234.123.41.44
[group0:vars]
var0=00
var1=01

[group1]
234.123.41.45
[group1:vars]
var0=10
var1=11

Now, in this configuration, variables have their values intact. But when I refer with both groups to the same host, like:

[group0]
234.123.41.44
[group0:vars]
var0=00
var1=01

[group1]
234.123.41.44
[group1:vars]
var0=10
var1=11

which I guess is rather typical when deploying to development environment, then the group0 variables get overwritten!

Since my configuration with "defaults" has been applied to similar inventory, my best guess is that this caused the problem there too. Now - what to do in order to have the variables keep their values for different groups, even if the host happens to be the same? And shouldn't it be the case anyway?

[Igor Cicimov]

Well of course they get overwritten in the *same* play since all variables are *global* during the play execution, otherwise if you call group1 after group0 what will happen is group1 will get the variables from group0, which is not desirable, right?

What you need to do is rethink your playbooks and inventory structure, think of using multiple roles instead of single one with many tasks and keep the global variables names unique if you don't want them overwritten.

[silv...@srebrnysen.com]

> On 2015-12-13, at 01:07, Igor Cicimov <ig...@encompasscorporation.com> wrote:
>
> Well of course they get overwritten in the *same* play since all variables are *global* during the play execution,

Well, until yesterday I wouldn't say "of course". If I used host variables, then I would understand this as being "of course". I thought that group variables are applied to the group's host(s) upon processing. In the sense that if I don't process the other group, the variables would remain intact. But then, after writing the previous post, I dumped the 'hostvars' and only then realised what's going on.

> otherwise if you call group1 after group0 what will happen is group1 will get the variables from group0, which is not desirable, right?

Frankly - I would expect/desire that when I call group0, I get variables defined for group0, when I call group1, I get the values as defined for group1. Even if they apply to the same host in the end. And if I wanted them within the host scope, I would use host_vars rather then group_vars. I understand that this is somewhat more difficult to implement, because a kind of "group context" or "group scope" would need to be introduced to the Ansible's inner workings. From what I saw in the variables dump, nothing like that's been done so far. Seems like during play execution, everything's thrown into one basket (becomes global as you mentioned). This is surely easier to implement but in this case I find the group_vars concept of noticeably lower value and also susceptible to errors, unless one understands that they are actually kind of "global host_vars".

> What you need to do is rethink your playbooks and inventory structure, think of using multiple roles instead of single one with many tasks

The setup is quite complex with many roles and even those roles have number of tasks, many of them being very similar, which is why tried to extract the repetitive tasks from within the roles and place those in the upper level, where it would be accessible for all roles. You know, the good, old DRY stuff.. "Of course" I used the same variable names in all the roles as they do the same thing and the shared, included tasks use them this way. Things - kind of - worked until I started including tasks, which is probably where it traverses the directory structure and overwrites the values.

> and keep the global variables names unique if you don't want them overwritten.

Actually - as I wrote above - I didn't expect them to be global. I wanted to keep them within the scope of role/group. Both approaches fail due to variables being overwritten. I spent several days on this already and eventually ended up generating different role/group variable naming (prefixes). Then I assign them (set_fact:) to the required variables right before including the tasks from upper level. Reworked half of the setup this way - works so far but I don't find it an elegant way. I've got this feeling like I am writing a program with tens of thousands lines of code and am allowed to use only global variables everywhere...

> --
> You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/gWN-uuj7NRI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/c74dc634-04fc-4380-8800-99b5978e649d%40googlegroups.com.