Exit with_items loop after first success?

[Hugh Saunders]

Hi All, 

I've been asking on IRC,  twitter, and github about this and was asked to post to the ML as well so here goes :)

I would like to be able to exit a with_items loop based on the result of the previous iteration. Think of it as a generalisation of with_first_found. 

Two use cases: 

1. I have a list of mirrors known to host file, I want to iterate over that list until I successfully retrieve the file, then stop iterating. No point in downloading the file multiple times.
2. I have a git ref (branch/tag) and a list of mirrors of that repo (git.openstack.org, github.com), I want to resolve that ref to a SHA without cloning the repo. Run git ls-remote via shell passing in each remote url in turn until the ref is successfully resolved. No point in resolving it twice. Note that for this use case I do not want to clone the repo. 

I'm sure there are other situations where it would be useful to break out of a loop. 

As explained in the issue, I can think of two ways of doing this, both would require modifications to ansible: 

1. Add previous_iteration variable for use in when clause, eg when: previous_iteration | failed
   
2. Allow registered variable to be used within a loop, eg when: shell_result['results'][-1].rc != 0
   

Thanks for any insight. 

--

Hugh Saunders

[Michael DeHaan]

"I would like to be able to exit a with_items loop based on the result of the previous iteration. Think of it as a generalisation of with_first_found. "

Would like to step back to use cases first before we propose it be done with the "with_<foo>" here, as that's not how that part works.

Curious what kinds of files you are downloading - for instance, package managers like yum already do this, etc.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/be63ee7b-502b-40a8-a776-cf05d8534c1c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hugh Saunders]

On 11 December 2014 at 16:00, Michael DeHaan <mic...@ansible.com> wrote:

"I would like to be able to exit a with_items loop based on the result of the previous iteration. Think of it as a generalisation of with_first_found. "

Would like to step back to use cases first before we propose it be done with the "with_<foo>" here, as that's not how that part works.

Curious what kinds of files you are downloading - for instance, package managers like yum already do this, etc.

In this particular instance, I am downloading tars of git repos. 

See also the second use case, its a bit of an edge case but shows that a general answer may be more useful than fixing a specific module. For example the file download case could be solved by modifying they get_url module to take a list of sources, but that only solves one specific case. Add the functionality to core and it can be used with any module. 

I'm also not sure a new with_ is needed, maybe access to another variable within loop conditionals, or a new conditional such as break_when? 

--

Hugh Saunders

[Michael DeHaan]

if you are downloading tars of git repos from GitHub from lots of production servers, that seems to be a bit of a bad practice to me that assaults the mirror.

I would consider setting up a mirror of all that content on one server initially and then have your individual production nodes download off that box.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAAD%2BY3Vh7uwrAwTVVcDf-MkbciLOTn%3Dj-5WSbq9OQu%3D0puhR_Q%40mail.gmail.com.

[Hugh Saunders]

On 11 December 2014 at 16:18, Michael DeHaan <mic...@ansible.com> wrote:

if you are downloading tars of git repos from GitHub from lots of production servers, that seems to be a bit of a bad practice to me that assaults the mirror.

I would consider setting up a mirror of all that content on one server initially and then have your individual production nodes download off that box.

I haven't said that I had multiple servers downloading the same content. 

Anyway... is there a way of breaking out of a loop or would patches be accepted to provide such a mechanism? 

 

--

Hugh Saunders

[Michael DeHaan]

"Anyway... is there a way of breaking out of a loop or would patches be accepted to provide such a mechanism?"

Still want to understand your use case before we start talking implementation and language changes to find the best way to express the construct.

Ultimately Ansible isn't a arbitrary scripting language, and I don't want to make it one - but we do want to find the best possible way to express what you may want to express.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAAD%2BY3VzJH1cLe6QOBC%3DQ9z_YtZSes15Q0o%2BPj-Zd%2BEKgqKXFA%40mail.gmail.com.

[Vladislav Rastrusny]

My use case is the following:

I have a list of ntpd servers and I want to run ntpdate against the list. But since one sucessful sync is enough, I would like to break the cycle after the first successfully executed sync. How would I do that currently? As I understand do-until loop will not work in this case, right?



четверг, 11 декабря 2014 г., 19:41:11 UTC+3 пользователь Michael DeHaan написал:

[Pete Ahearn]

Bump. Is there really no way to do this currently?

[Chuck Carlino]

https://groups.google.com/forum/#!topic/ansible-project/z9iD55-4pPs is a similar case.

[Jonathan]

I'd also be super interested in some way to do this.  To me this is a very sane use case, and the Ansible appears to allow the use of until: along with with_items:, but appears to just ignore the until statement.  If this is not supported ideally there would be some sort of syntax issue.

Would the Ansible team be willing to take a pull request to allow until evaluations in the context of a with_items loop?

For more clarity here is an example play.  Lets say we want to send a update to a single web server out of a set, and you want to stop after the first successful call.  Maybe the calls are not idempotent, maybe you don't want to add extra load...

here is a test play with until and with_items, and until apparently not being evaluated.

- hosts: localhost

  gather_facts: no

  tasks:

  - name: test

#    action: debug msg="cluster host {{ item }}"

#    action: uri timeout=2 url={{ item }} 

    action: shell curl --max-time 1 {{ item }} >/dev/null

    with_items: "{{input_hosts|shuffle}}"

    register: put_status

    until: put_status.rc == 0

here is the output executing with an input array of URLs, you can see the first URL works yet it keeps going

ansible-playbook -vv -i ../hosts --sudo test2.yml --extra-vars='{"input_hosts": ["https://www.google.com", "https://www.snargoblarg.com"]}'

PLAY [localhost] ************************************************************** 

TASK: [test] ****************************************************************** 

<127.0.0.1> REMOTE_MODULE command curl --max-time 1 https://www.google.com >/dev/null #USE_SHELL

changed: [127.0.0.1] => (item=https://www.google.com) => {"attempts": 0, "changed": true, "cmd": "curl --max-time 1 https://www.google.com >/dev/null", "delta": "0:00:00.555617", "end": "2016-01-04 23:32:04.951993", "item": "https://www.google.com", "rc": 0, "start": "2016-01-04 23:32:04.396376", "stderr": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100 18908    0 18908    0     0  35322      0 --:--:-- --:--:-- --:--:-- 36152", "stdout": "", "warnings": ["Consider using get_url module rather than running curl"]}

<127.0.0.1> REMOTE_MODULE command curl --max-time 1 https://www.snargoblarg.com >/dev/null #USE_SHELL

<127.0.0.1> REMOTE_MODULE command curl --max-time 1 https://www.snargoblarg.com >/dev/null #USE_SHELL

Result from run 1 is: {'cmd': 'curl --max-time 1 https://www.snargoblarg.com >/dev/null', 'end': '2016-01-04 23:32:11.354140', 'stdout': u'', 'changed': True, 'attempts': 1, 'start': '2016-01-04 23:32:11.188499', 'delta': '0:00:00.165641', 'stderr': ' % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: www.snargoblarg.com', 'rc': 6, 'warnings': ['Consider using get_url module rather than running curl']}

<127.0.0.1> REMOTE_MODULE command curl --max-time 1 https://www.snargoblarg.com >/dev/null #USE_SHELL

Result from run 2 is: {'cmd': 'curl --max-time 1 https://www.snargoblarg.com >/dev/null', 'end': '2016-01-04 23:32:16.868708', 'stdout': u'', 'changed': True, 'attempts': 2, 'start': '2016-01-04 23:32:16.703207', 'delta': '0:00:00.165501', 'stderr': ' % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: www.snargoblarg.com', 'rc': 6, 'warnings': ['Consider using get_url module rather than running curl']}

<127.0.0.1> REMOTE_MODULE command curl --max-time 1 https://www.snargoblarg.com >/dev/null #USE_SHELL

Result from run 3 is: {'cmd': 'curl --max-time 1 https://www.snargoblarg.com >/dev/null', 'end': '2016-01-04 23:32:22.253845', 'stdout': u'', 'changed': True, 'attempts': 3, 'start': '2016-01-04 23:32:22.231865', 'delta': '0:00:00.021980', 'stderr': ' % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: www.snargoblarg.com', 'rc': 6, 'warnings': ['Consider using get_url module rather than running curl']}

failed: [127.0.0.1] => (item=https://www.snargoblarg.com) => {"attempts": 3, "changed": true, "cmd": "curl --max-time 1 https://www.snargoblarg.com >/dev/null", "delta": "0:00:00.021980", "end": "2016-01-04 23:32:22.253845", "failed": true, "item": "https://www.snargoblarg.com", "rc": 6, "start": "2016-01-04 23:32:22.231865", "warnings": ["Consider using get_url module rather than running curl"]}

stderr:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: www.snargoblarg.com

msg: Task failed as maximum retries was encountered

FATAL: all hosts have already failed -- aborting

[Dayton Jones]

I have need of this feature too, would be quite beneficial to some tasks I need.

[Ivan Ogai]

I also need this feature.

[Davide Scrimieri]

This feature would be really helpful. Are there any news on this?

[Nikolay Dimov]

This a very important feature to have, and not only on success  but on any condition (e.g. on failure). Currently, it just keeps running until it finishes the loop which is not always a desired outcome.

[Nikolay Dimov]

I ended up using the following hack:

- name: shell test

shell: "{{ item }}"

with_items:

- /bin/true

- /bin/false

- /bin/true

when: r_shell.rc|default(0)==0

register: r_shell

This exits on first failure. To exit on success you can use:

when: r_shell.rc|default(1)!=0

This is obviously a hack, and it is strange that register looks different in the task and after the task ends, but it does the job.

It would have been great if the register had the final form and the item results were appended as tasks were executed so we can check the results there with filters.

[Stefan Hornburg (Racke)]

On 5/16/20 2:12 PM, Nikolay Dimov wrote:
> This a very important feature to have, and not only on success  but on any condition (e.g. on failure). Currently, it
> just keeps running until it finishes the loop which is not always a desired outcome.
>

Hello Nikolay,

I never needed this very important feature, so what is your actual use case?

Regards
Racke

> --
> You received this message because you are subscribed to the Google Groups "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> ansible-proje...@googlegroups.com <mailto:ansible-proje...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/ansible-project/eefefd1a-40a7-4fd5-8a0c-9091fd76d0da%40googlegroups.com
> <https://groups.google.com/d/msgid/ansible-project/eefefd1a-40a7-4fd5-8a0c-9091fd76d0da%40googlegroups.com?utm_medium=email&utm_source=footer>.


--
Ecommerce and Linux consulting + Perl and web application programming.
Debian and Sympa administration. Provisioning with Ansible.

[Vladimir Botka]

On Sun, 17 May 2020 02:19:26 -0700 (PDT)
Nikolay Dimov <dimo...@gmail.com> wrote:

> I ended up using the following hack:
>
> - name: shell test
> shell: "{{ item }}"
> with_items:
> - /bin/true
> - /bin/false
> - /bin/true
> when: r_shell.rc|default(0)==0
> register: r_shell
>
> This exits on first failure.

No. It does not. This does not solve the problem that the loop "just keeps
running until it finishes the loop". The loop will run further after the
failure and the remaining items will be skipped because of the condition.

Then, because of the failure, the playbook will be terminated.

> To exit on success you can use:
>
> when: r_shell.rc|default(1)!=0

The result of this condition is completely different. The first successful
iteration makes all the rest to be skipped. But, because there was no
failure, the playbook will continue. Right?

HTH,

-vlado

[Brian Coca]

It is something i would like to add as part of a full looping revamp
https://github.com/ansible/proposals/issues/140




--
----------
Brian Coca