extra_vars don't override inventory variables in templates (continuing #9242)

[Hagai Kariti]

Hi

I hope this is the right place to post this. I reported https://github.com/ansible/ansible/issues/9242 a few days ago and I still have issues after it was closed. I'll give a similar example to the one I gave in the issue, but using vars_files and adding one more usecase:

inventory:

localhost ansible_connection=local

[all:vars]

var=one

vars_file:

var2: "{{var}}"

mapping:

  one: 1

  two: 2

playbook:

---

- hosts: localhost

  gather_facts: no

  vars_files:

    - vars_file

  vars:

    value: "{{mapping[var]}}"

  tasks:

    - debug: var=var

    - debug: var=var2

    - debug: var=value

command:

    ansible-playbook -i inventory playbook.yml -e var=two

output:

TASK: [debug var=var] ********************************************************* 

ok: [localhost] => {

    "var": "two"

}

TASK: [debug var=var2] ******************************************************** 

ok: [localhost] => {

    "var2": "one"

}

TASK: [debug var=value] ******************************************************* 

ok: [localhost] => {

    "value": "1"

}

In older versions (1.5.5 is the one I test with), the third (mapping) usecase is working properly, but the second does not. In the latest dev, both don't work 

Thanks!

[Hagai Kariti]

Another case, which isn't related to extra_vars but to the root cause (according to my best guess) of evaluating the variables expressions too early:

inventory:

localhost ansible_connection=local

[all:vars]

var=value

Playbook:

---

- hosts: localhost

  gather_facts: no

  vars:

    var2: "{{ var  | default('none') }}"

  tasks:

    - debug: var=var

    - debug: var=var2

Output:

TASK: [debug var=var] ********************************************************* 

ok: [localhost] => {

    "var": "value"

}

TASK: [debug var=var2] ******************************************************** 

ok: [localhost] => {

    "var2": "none"

}

[Hagai Kariti]

Since my issue was closed without a response and redirected me to the list, is it possible to get a response here?

On Tuesday, October 7, 2014 10:50:13 AM UTC+3, Hagai Kariti wrote:

[Stephen Gargan]

Hagai,

I had a similar issue and posted a pull request for it a while back. Its not been merged yet but you can grab it here 

https://github.com/ansible/ansible/pull/9106

Try applying it and see if it fixes your issue too.

regards,

Steve.

[Hagai Kariti]

Hi Stephen

First of all, thanks a lot. Your changes however did not solve my second case, the one using 'default' on inventory variables. So I guess some more tweaking is required. I'd love to help btw.

Something is also weird about your pull request - it's not in its own branch, which makes it more difficult to pull to my repo (I had to copy the individual files manually!).  I put them in a branch in my fork if you want: https://github.com/hkariti/ansible/tree/premature_template_vars

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/EunpeCL45k8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/12a88f83-d97f-4afe-93b3-644ee6887180%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael DeHaan]

Trying to sort through the email storm here :)

Yes, vars_files and such will evaluate all the variables it can *So that they may be used* if possible early, as that is often the case.

While it doesn't help your question, I would try to not complicate your infrastructure so much, it's hard to say from your theoretical example, but if you could post a more real world example of showing what you are trying to model, I think you might be doing something a little non-idiomatic.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAO0%3DbmHkhDRcMx9DcuvRNLjneiksMsvSmRf9f3EBJEZFDgdGVA%40mail.gmail.com.

[Hagai Kariti]

Hi Michael, thanks for your response.

Before I describe my usecase I'd like to point out that the current behavior is a bug IMO, because it evaluates variables before all variable sources are looked at. I understand evaluating variables early so they can be used as soon as possible, but no task/play is run before all var sources are loaded anyway, so I don't see a case when a variable is unavailable if evaluating is postponed in this case.

Now for my usecase. I have a play that creates ec2 instances. An ec2 instance needs a region to be created in, and a base image id (AMI id in ec2 terms).

I defined the default region to create instances as an inventory variable (as it's different for prod and stage environments):

prod/group_vars/all:

    region: us-west-1

and I have a region<->AMI id mapping in an external vars file:

    amis:

        us-west-1: AMI1

        us-east-1: AMI2

And the play uses it like so:

    vars:

       ami_id: "{{amis[region]}}"

Problem is, if I want to run the playbook and override the region with -e I can't, because ami_id is evaluated before '-e' takes effect.

I have a different usecase for my second example (the once with the |default line) if you want. If you think I didn't model this case well, I'll be glad to hear of a better way that solves my problem. Nevertheless, I still think one should expect the above way to work.

Thanks again.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgxuB2GUWp-W3LpLnO_utLo-wLMadcFyxVhM5u%2BdaSdqXQ%40mail.gmail.com.

[Michael DeHaan]

-e does bind pretty early, but it seems like you are using variables in paths to "vars_files" that already resolve, so it selects what file to use earlier.

This is technically required to set globals in those files.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAO0%3DbmHw8%2BkdCJCkgHfc1j7ujF_7eU%2BE49WKEHaQeSMZVarZSA%40mail.gmail.com.

[Hagai Kariti]

On Oct 30, 2014 5:53 PM, "Michael DeHaan" <mic...@ansible.com> wrote:
>
> -e does bind pretty early, but it seems like you are using variables in paths to "vars_files" that already resolve, so it selects what file to use earlier.
>

I don't, actually. I don't template the vars_files path at all. All I do is use a variable that's defined in a vars_file in a regular vars section.

> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgyKCSB5Rv0mEMM8kmethk8yjoroJcekQUVWymu%2BXb0OAw%40mail.gmail.com.

[Hagai Kariti]

Ok, I poked around a bit and indeed found one precedence issue and one premature templating issue:

In lib/ansible/runner/__init__.py lines 612-613:

        module_vars_inject = utils.combine_vars(host_variables, combined_cache.get(host, {}))

        module_vars_inject = utils.combine_vars(self.module_vars, module_vars_inject)

module_vars are overrided by host_variables, although they are supposed to be of high precedence. They are being used in the correct order (module_vars > host_variables) when building the inject var. Changing the order of precedence in the above lines solves my first use-case (the mapping override one).

In lib/ansible/playbook/play.py lines 89-90:

        all_vars = utils.combine_vars(self.vars, self.playbook.extra_vars)

        self.vars = template(basedir, self.vars, all_vars)

play vars are templated without taking inventory variables into account. Removing these lines solves my second use-case (defining a variable in vars section based on inventory var with a default value).

make tests seem to pass. Is there a chance that someone with more knowledge of how vars behave can verify if my fixes are correct? Should I open a PR for this?

Thanks

[Michael DeHaan]

Would need to see your complete playbook.

Definitely don't modify the code just yet :)

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/591094e7-c26d-4fe0-a9c2-e8c34a9f8160%40googlegroups.com.

[Hagai Kariti]

I gave two example playbooks in this thread and in the github issue (same ones). Do you want a more 'real world' example?