When statements with Jinja2 templating delimiters

[Rich Lees]

Hi All,

Am new to this group and Google Groups in general so please go easy.

I have a few questions about the recent change in Ansible 2.3.0.0 that new prints a warning when jinja2 templating is included in a when, changed_when and failed_when statement. Please see this closed GitHub issue for more details on exactly what warning I mean: https://github.com/ansible/ansible/issues/22397

We have some very complex plays that depend on multiple roles. The roles are kept very generic by design so they can be used by multiple calling roles. Example: we may have plays for three or four completely different Java web apps that all depend on a common Tomcat role, passing in overriding variables for customisation.

This now causes any when statement that uses one of these overridden variables to throw a warning. Please find a very simplistic example below:

Parent Role - defaults

bootstrap_ping : true

Parent Role - meta

dependencies:

  - { role: apollo/platform/ping, ping: '{{ bootstrap_ping }}' }

Child Role - tasks

- name: execute

  when: ping

  win_ping:

This throws the warning.

I have looked at a few GitHub threads and a few Stack Overflow questions and have yet to see a valid workaround for this situation.

I also do not understand why Ansible processes Jinja2 under when statements if the stance is you are not supposed to do it. I raised a PR (here: https://github.com/ansible/ansible/pull/24974) which was swiftly closed (fair enough, I have never contributed to this project before and I did not follow procedure).

I was hoping someone could please help with the below questions to educate myself and improve our Ansible playbooks further:

1. Will this remain forevermore just a warning or will it become a hard, breaking error in a future release?
2. What is the 'proper' way to write the above example whilst keeping things modular?
3. If this will remain just as a warning what about the possibility of implementing an ansible.cfg flag to turn it off?

Really appreciate any help,

Thanks!

Rich

[Micheal Cottingham]

My short answer: Yes.


I mentioned on GitHub that I'm new the Ansible and just starting to learn it. It caused a great deal of confusion for me, when I saw that everywhere else in Ansible accepts Jinja delimeters but `when` and `changed_when` threw these warnings. This is a rather simplistic example of the frustration trying to get the warning to go away. It may not be the ideal way to fix it, but that's also my point - I just needed to drop in a variable to be parsed and I ended up having to concatenate strings to get it to work:

- name: "Dokku Plugins: Install"
  become: true
  command: "dokku plugin:install https://github.com/dokku/dokku-{{ item }}.git"
  args:
    creates: "/var/lib/dokku/plugins/available/{{ item }}/*"
  register: dokku_plugin_install
  changed_when: dokku_plugin_install.stdout|search('Plugin ' + item + ' enabled')
  tags:
    - dokku_install_plugin
  with_items:
    - mysql
    - letsencrypt
    - maintenance


Hopefully pretty straight-forward on what I'm trying to solve. The quick explanation is I'm installing some plugins in Dokku using the command "dokku plugin:install" and providing it with a loop of items. Then to check whether it is already installed, I search stdout.

In this case, I may have gotten away with searching for "enabled." But I lean towards being explicit so I'm clear in what's going on. Ultimately, I'd like to release this on Galaxy for people to use. So I'd like to make it clear what it is that I'm doing in my tasks.

However, as I continue to build out my roles and tasks, I may not always have the option to do avoid using variables.

While my use case is a bit different from Rich's use case, I think what it comes down to for both of us is lack of clarity and consistency in how the Jinja templates work.

[Brian Coca]

@Micheal, in your case having the extra moustaches ({{ }}) would cause
unwanted double interpolation, @Rich is actually using that on
purpose. So the warning was valid in your case, but not his.

For templating there are mainly 2 rules (though 2nd is actually
consequence of the first):

- moustaches don't stack: {{ {{ }} }} <= you should not use tempating
inside templating, normally you want contactenation or hostvars[]
- Always use moustaches except when `when:` <= conditionals are
'expressions' so they already have an implied template which then hits
rule #1



----------
Brian Coca

[Micheal Cottingham]

Hi,

Thanks for getting back to me. I understand that in my case it is a valid warning for what it was designed for. Where I disagree is the warning being present in the first place.

I admit that my use case is different, but as a new person to Ansible, it is confusing why there's a lack of consistency between when and changed_when and the rest of Ansible.

I do agree that my usage would result in double interpolation and I did consider that. But I wanted, and perhaps poorly explained, to take a step up a level to the lack of consistency as I see it with the rest of Ansible.

As I mentioned, I'm new to Ansible. But this seeming lack of consistency between Jinja delimiters is what caused my frustration. I expected to be able to use Jinja delimiters and was surprised when I couldn't.

I hope that makes more sense on what I'm looking for. 

[Brian Coca]

There are just 2 contexts, normal (you require them) and conditionals
(anything with when:) which don't. It is consistent, juts not uniform.

----------
Brian Coca

[Micheal Cottingham]

That's fair. Thanks for the details.

[Jörg Kastning]

Am Mittwoch, 24. Mai 2017 17:03:20 UTC+2 schrieb Rich Lees:

Hi All,

Am new to this group and Google Groups in general so please go easy.

I have a few questions about the recent change in Ansible 2.3.0.0 that new prints a warning when jinja2 templating is included in a when, changed_when and failed_when statement. Please see this closed GitHub issue for more details on exactly what warning I mean: https://github.com/ansible/ansible/issues/22397

Hi All,

I found this discussion in the githup issue mentined above. In my case I need to run a play only when a certain string was found an the output of another command. Example main.yml from one of my roles:

---
  - name: Install Red Hat Security Advisory (RHSA)
    command: yum -y update-minimal --advisory {{ rhsa_to_install }}
    register: yum_output
  - debug: var=yum_output

  - name: Reboot Host if any packages were updated
    shell: sleep 2 && shutdown -r now "Ansible updates triggered"
    async: 1
    poll: 0
    ignore_errors: true
    when: ("Complete!" in "{{ yum_output.stdout_lines[-1] }}") or
          ("Komplett!" in "{{ yum_output.stdout_lines[-1] }}")

 Since Ansible 2.3 I got the warnings described in detail in https://github.com/ansible/ansible/issues/22397.

I would like to get back the behaviour from Ansible <2.3. While I'm new to Ansible I may simply not know how to work around this warnings by declaring the when statement in a different way. Could someone give me a hand here?

Regards,
Joerg

[Kai Stian Olstad]

On 26. mai 2017 14:54, Jörg Kastning wrote:
> I found this discussion in the githup issue mentined above. In my case I
> need to run a play only when a certain string was found an the output of
> another command. Example main.yml from one of my roles:
> ---
> - name: Install Red Hat Security Advisory (RHSA)
> command: yum -y update-minimal --advisory {{ rhsa_to_install }}
> register: yum_output
> - debug: var=yum_output
>
> - name: Reboot Host if any packages were updated
> shell: sleep 2 && shutdown -r now "Ansible updates triggered"
> async: 1
> poll: 0
> ignore_errors: true
> when: ("Complete!" in "{{ yum_output.stdout_lines[-1] }}") or
> ("Komplett!" in "{{ yum_output.stdout_lines[-1] }}")
>
> Since Ansible 2.3 I got the warnings described in detail in
> https://github.com/ansible/ansible/issues/22397.
>
> I would like to get back the behaviour from Ansible <2.3. While I'm new to
> Ansible I may simply not know how to work around this warnings by declaring
> the when statement in a different way. Could someone give me a hand here?

In your case you need to do what the warning says, remove the "{{ and
the }}" and the warning will be gone.

when: ("Complete!" in yum_output.stdout_lines[-1]) or
("Komplett!" in yum_output.stdout_lines[-1])

--
Kai Stian Olstad

[Jörg Kastning]

Thanks for helping me Kai. I appreciate it.

This change took me by surprise and I'm lucky that I only have a few roles to change. IMHO it was easier to read with the curly braces, because you could see at the first look that there is a variable substitution taking place.

Brian wrote:

There are just 2 contexts, normal (you require them) and conditionals
(anything with when:) which don't. It is consistent, juts not uniform.

I do not understand the need for two different contexts here? Why could it not kept the way like it was in Ansible <2.3.

Thanks,
Joerg

[Kai Stian Olstad]

On 27. mai 2017 13:40, Jörg Kastning wrote:
> Am Freitag, 26. Mai 2017 18:12:43 UTC+2 schrieb Kai Stian Olstad:
>> In your case you need to do what the warning says, remove the "{{ and
>> the }}" and the warning will be gone.
>>
>> when: ("Complete!" in yum_output.stdout_lines[-1]) or
>> ("Komplett!" in yum_output.stdout_lines[-1])
>>
>
> Thanks for helping me Kai. I appreciate it.

You're welcome.

> Brian wrote:
>
>> There are just 2 contexts, normal (you require them) and conditionals
>> (anything with when:) which don't. It is consistent, juts not uniform.
>
>
> I do not understand the need for two different contexts here? Why could it
> not kept the way like it was in Ansible <2.3.

In some other context that way did cause errors.

If the Jinja template context was removed from when your way of writing
it would fail. The reason for this is that "string" in a.variable is a
Jinja expression
http://jinja.pocoo.org/docs/dev/templates/#other-operators

If removed you would have to add the {{ at the beginning and }} at the
end like every time like so.

when: "{{ ("Complete!" in yum_output.stdout_lines[-1]) or
("Komplett!" in yum_output.stdout_lines[-1]) }}"

--
Kai Stian Olstad

--
Kai Stian Olstad

[Rich Lees]

Hi Micheal,

Thanks for the reply. I have also seen pretty much that any other example of this warning being triggered is both valid and workaround-able.

I am beginning to think our way of using Ansible (passing through variables into sub-roles) might be different to other peoples...

Which brings me back to my original questions. Will this always be a warning or will it become a "hard" error? At this point I understand that the change won't be reverted and why I just would like to know how big the impact of this will be in the future.

[Rich Lees]

I mean Hi Brain! ooops... 

I have no idea how to edit posts either!

[Ed]

[ Bringing a question over here that was unanswered in ansible-project... ]  I've got cases where I want to run the same complex task multiple times, with minor variances.  I've been implementing that with a loop where each item has a property for its when statement, and the task uses templating to resolve the property.  With 2.3, I now receive warnings about the jinja expression.  I can't use "when: item.condition", since that handles item.condition as a string (instead of evaluating the content of item.condition).  What's the best way to accomplish this, warning-free, in 2.3?  Is there a filter I can use to do evaluation?  For maintenance reasons, I'd like to avoid building a distinct task for each case.

Simple example below.

Thanks,

Ed

$ cat inventory

one ansible_connection=local

$ cat play.yml

---

- hosts: all

  tasks:

    - debug:

        msg: "{{ item.message }}"

      when: "{{ item.condition }}"

      with_items:

        - { message: "hello", condition: "7 < 4" }

        - { message: "goodbye", condition: "7 > 4" }

$ ansible-playbook -i inventory play.yml

PLAY [all] *******************************************************************************

TASK [Gathering Facts] *******************************************************************

ok: [one]

TASK [debug] *****************************************************************************

 [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }}

or {% %}. Found: {{ item.condition }}

ok: [one] => (item={u'message': u'goodbye', u'condition': u'7 > 4'}) => {

    "item": {

        "condition": "7 > 4",

        "message": "goodbye"

    },

    "msg": "goodbye"

}

skipping: [one] => (item={u'message': u'hello', u'condition': u'7 < 4'})

PLAY RECAP *******************************************************************************

one                        : ok=2    changed=0    unreachable=0    failed=0

$

[Rich Lees]

Latest devel seems to work great for me now. Thanks to the developers for fixing it for this specific use-case.

Our codebases no longer generate any warnings except one or two that are both valid and very easy to fix

Thanks!

[Danny Cosson]

Rich - what was the particular case that was fixed in the latest devel? Do you (or anyone else) have a link to the PR?

I'm running into the exact issue you describe when trying to keep my ansible roles modular (i.e. passing boolean values in as arguments to the role instead of relying on the global fact values). I haven't tried the devel branch.

A few comments here addressed the confusion this caused, but IMO this was a pretty dangerous warning in this scenario. If you blindly listen to the warning and remove the {{ }} jinja2 interpolation, there's a good chance it will still run but it now uses the name of the variable as a string value so the behavior will be totally different as it won't match any when conditions (neither as true or false).

Anyway, hopefully it's resolved in the next release?

[andre...@kiwigrid.com]

I still run in this problem with Ansible 2.4.1.0.

Which version are you refering to?