Running tasks in parallel in a playbook

[CS]

Is there a way to execute tasks in a playbook in parallel on a single host? I expected something like the following to work based on the docs:

---
- name: deploy
  hosts: $target
  gather_facts: false
  tasks:
  - name: sleep 10
    action: command /bin/sleep 10
    async: 11
    poll: 5
  - name: sleep 10 again
    action: command /bin/sleep 10
    async: 11
    poll: 5

I run that with a command like:

ansible-playbook deploy.yml -f 10 -e "target=`hostname`" --connection=local

I thought the two sleep tasks would happen in parallel, but that doesn't happen. It takes about 23 seconds to complete, and 'ps' shows that they run serially. If I set 'poll: 0' then they run in parallel, but I lose the ability to know if they completed successfully or not, and the playbook finishes while they're still running, which is not what I want.

Is there some way to get tasks in a play (or even in separate plays within a single playbook) to run in parallel while also waiting for successful completion (up to the timeout)?

Thanks!

[Michael DeHaan]

No, there's really nothing for parallel jobs on the same host unless you did "fire and forget" (async without poll).  You might look at something like calling gnu paralell inside a script at that point if you wanted to wait on all of them.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[CS]

Thanks for the response. I thought of using bash `wait` and a script, but I would like to stay in Ansible as much as possible.

My use case is that there are multiple "slow enough" independent tasks that happen on a host, but I can't fire and forget because I need to know that they either succeed or fail. That seems like a pretty common use case, and I have other things too that would benefit from this kind of parallelism, but maybe I'm using Ansible in a non-standard way. In this case, I'm installing N (2 or 3 for now) python virtualenvs for independent services on a single host, each of which takes about 5 minutes to install. I'd love to have developers wait 5 minutes rather than 5N minutes for that part of the installation, and there are other tasks too that nothing else depends on but that need to be reported as succeeding or failing.

Are there any other approaches I might consider using Ansible apart from moving all the parallel tasks into a script?

[benno joy]

Probably you can fire and forget and at the end of the play, write a task that checks the status of parallel tasks via checking a port or reading from a logfile etcc, and then take action as necessary.

Regards,

Benno

[Jesse Keating]

On Aug 28, 2013, at 9:52 AM, CS <sapien...@gmail.com> wrote:
>
> Thanks for the response. I thought of using bash `wait` and a script, but I would like to stay in Ansible as much as possible.
>
> My use case is that there are multiple "slow enough" independent tasks that happen on a host, but I can't fire and forget because I need to know that they either succeed or fail. That seems like a pretty common use case, and I have other things too that would benefit from this kind of parallelism, but maybe I'm using Ansible in a non-standard way. In this case, I'm installing N (2 or 3 for now) python virtualenvs for independent services on a single host, each of which takes about 5 minutes to install. I'd love to have developers wait 5 minutes rather than 5N minutes for that part of the installation, and there are other tasks too that nothing else depends on but that need to be reported as succeeding or failing.
>
> Are there any other approaches I might consider using Ansible apart from moving all the parallel tasks into a script?

What I've done to accomplish this is to create fake inventory entries, one entry per action I want to do in parallel. Then I have a play that works over this group of "hosts" to execute the action(s) I want done, all delegated to localhost.

This kicks in Ansible's forking and lets the tasks run in parallel.

-jlk

[Michael DeHaan]

Something more native might be nice :)

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Bob Tiernay]

Something native to Ansible would be great. I can see this being generally useful and a feature I'm certainly interested in.

[Michael DeHaan]

So, this is an old post, so I have an update for you.

You can already get a fair measure of this if you are an AWX user, here to launch a few Job Templates by ID:

awx-cli joblaunch -t 0 && awx-cli joblaunch -t 1

This also gives you central status logging and access control -- delegation around who can deploy to what, and is *super super super* trivial to wire up into something like Jenkins to achieve continuous deployment practices (bonus points for using our load balancer modules for rolling updates)

The AWX cli is of course a stub, but you can easily see how to do more with it, including poll for job completion and search what jobs are available, etc.

https://github.com/ansible/awx-cli/

http://www.ansibleworks.com/ansibleworks-awx/

On Tue, Dec 10, 2013 at 12:01 AM, Bob Tiernay <rtie...@gmail.com> wrote:

Something native to Ansible would be great. I can see this being generally useful and a feature I'm certainly interested in.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Giovanni Gaglione]

Hi Jesse,

Could you make an example of this solution?

My scenario is like the following:

[myhost]

192.168.1.1

192.168.1.1
192.168.1.1

192.168.1.1
192.168.1.1
192.168.1.1
192.168.1.1

But then, when ansible iterates over these host, I would like to have a unique name of the host (example `192.168.1.1-3th`). It seems there is not way to get the index of the host.

Any suggestion? 

[Giovanni Gaglione]

I basically need the index of the current hostname, but I can't find a way. 

{{ groups['browsers'].index(inventory_hostname) }} will return always the same index (because the hostname is always the same).

I couldn't find any other way. Do you know if the API offers something more appropriate? Still doing research...

[Giovanni Gaglione]

I've solved it this way - https://github.com/ansible/ansible/issues/12086#issuecomment-327927275

[Dick Davies]

http://docs.ansible.com/ansible/latest/playbooks_loops.html#looping-over-a-list-with-an-index

On 7 September 2017 at 19:14, Giovanni Gaglione

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.

> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/76c1c632-7921-4fa3-97ce-77d9e27d475d%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.