Can we run roles in parallel for multiple hosts?

[Shweta Krishnan]

New ansible user here.

After deploying VMs, I want to run a folder called configs on the Jump server to components like Syslog, NTP etc. for post configurations. 

This is the structure of how files are in the config folder

configs/ 

├─ hosts 

├─ group_vars 

├─ playbook.yml 

├─ roles/ 

       ├─ Ntp/

               ├─ tasks/  

               ├─ vars/ 

       ├─ Syslog/  

               ├─ vars/ 

               ├─ tasks/

This is how it is my main playbook - playbook.yml

- name: Setup config for ntp

 hosts: comp1 

 roles: - comp1 

 - name: Setup config for syslog

 hosts: comp2 

 roles: - comp2

Right now I run this command 

ansible-playbook -i hosts playbook.yml --limit "ntp,syslog"

this runs the tasks sequentially. How can I execute this in parallel considering there is no dependency between the components?

[David Logan]

Morning Shweta,

This is documented here https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.html

Regards

David

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/835db2cd-ba3f-4987-abef-98486d068b96n%40googlegroups.com.

--

if in trouble, or in doubt
run in circles, scream and shout

[Shweta Krishnan]

Hello David!

Thank you for the link. Would this strategy and fork option work for roles as well? 

In the user guide, there was no mention of roles, that's why I asked.

[David Logan]

I put the setting in my ansible.cfg. I tend to use small roles and import each role as needed in a playbook so I'm not sure how that would work with just roles. If you are trying to use RH Satellite then that has a different ansible.cfg than the standard (in /etc/foreman-proxy/ansible.cfg)

You can also put it in the playbook

  hosts: all
  strategy: free

I put the forks in the command-line as --forks 50 or whatever. There is a small calculation that you can find on line (basically ncpus * 4 = number of forks but also if you have a chunk of memory then you can increase it)

Regards

David

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/88ca0901-4bb6-45a5-ab00-9deb908c1cacn%40googlegroups.com.

[David Logan]

I reread your original post, I think you will find each task/role within the playbook will be run sequentially. The strategy links that I gave you will help with getting them running on multiple hosts in roughly parallel. I'm unsure if you can parallelize the tasks.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/88ca0901-4bb6-45a5-ab00-9deb908c1cacn%40googlegroups.com.

[dulh...@mailbox.org]

wondering about the 'why' of your question I I guess it is about speeding up execution?

There is an extension mitogen4ansible which claims to speed up playbook execution significantly. If I understand that correctly it is related to parallelizing things in some way on a python level. Here is some youtube food for thought on this

[Marc Haber]

On Mon, Jun 27, 2022 at 05:04:38PM +0200, dulhaver via Ansible Project wrote:
> There is an extension mitogen4ansible

> <https://aperogeek.fr/ansible-mitogen/> which claims to speed up playbook
> execution significantly.

Using mitogen is a no-brainer to sped up ansible runs. I was able to
reduce my ansible running times to like a fifth of the time running with
stock ansible with ZERO changes to my roles, tasks, playbooks and no
additional software or configuration needed on the targets. It all
happens on the controller and is enabled in a minute.

Highly recommended!

Greetings
Marc

--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421

[Michael Ströder]

On 6/27/22 17:17, Marc Haber wrote:
> On Mon, Jun 27, 2022 at 05:04:38PM +0200, dulhaver via Ansible Project wrote:
>> There is an extension mitogen4ansible
>> <https://aperogeek.fr/ansible-mitogen/> which claims to speed up playbook
>> execution significantly.
>
> Using mitogen is a no-brainer to sped up ansible runs.

I can confirm the speed-up achieved by mitogen but there are two issues:

1. Some tasks won't work properly, e.g. if you have to set
ansible_python_interpreter for a task.

2. IIRC it's not maintained anymore and it does not work with ansible 2.10+.

Ciao, Michael.

[Marc Haber]

On Mon, Jun 27, 2022 at 05:56:01PM +0200, 'Michael Ströder' via Ansible Project wrote:
> 2. IIRC it's not maintained anymore and it does not work with ansible 2.10+.

This might be weirdness with Debian version numbers, but my ansible
installed from debian is Version 5.5.0, and it works great with mitogen:

| [4/5063]mh@drop:~ $ dpkg --list '*ansible*' | grep -v '^un'
| Desired=Unknown/Install/Remove/Purge/Hold
| | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
| |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
| ||/ Name Version Architecture Description
| +++-===============-============-============-===============================================================
| ii ansible 5.5.0-1 all Configuration management, deployment, and task execution system
| ii ansible-core 2.12.4-1 all Configuration management, deployment, and task execution system
| ii ansible-mitogen 0.3.2-1 all Fast connection strategy for Ansible
| [5/5064]mh@drop:~ $

[Michael Ströder]

On 6/27/22 21:59, Marc Haber wrote:
> On Mon, Jun 27, 2022 at 05:56:01PM +0200, 'Michael Ströder' via Ansible Project wrote:
>> 2. IIRC it's not maintained anymore and it does not work with ansible 2.10+.
>
> This might be weirdness with Debian version numbers, but my ansible
> installed from debian is Version 5.5.0, and it works great with mitogen:
>

> | ii ansible 5.5.0-1 all Configuration management, deployment, and task execution system
> | ii ansible-core 2.12.4-1 all Configuration management, deployment, and task execution system
> | ii ansible-mitogen 0.3.2-1 all Fast connection strategy for Ansible

Glad to see that there are mitogen updates. Thanks for the heads up.

Currently it seems to not work with ansible-core 2.13.x though. But
hopefully there will be an update soon.

Ciao, Michael.

[Shweta]

Hello, I tried mitogen for the execution of some other ansible script and saw a significant difference in the time taken to complete the play. So thank you for this tool!

But is there no way to execute these roles in parallel? 

As I described before as well - 

My main playbook.yml file calls role-wise to initiate setup and then inside roles I have tasks for different components.  All hosts are different and execution is independent of each other. 

[Brian Coca]

Yes and No, you can run playbooks in parallel, you cannot run roles in
parallel in the same play nor playbook.

While all tasks run in parallel and you can do things like async or
free strategy this wont parallelize the roles themselves, they get
executed in the order in which they are address in the same play. So
if you want to run roles in parallel i suggest executing parallel
instances of ansible-playbook, parallel, xargs and other utilities
make this simple, or via a scheduler cron/incron/awx|tower|aap,
jenkins, etc). Theoretically you can write a strategy plugin that does
this, but this is far from trivial.

A note about mitogen, since it was mentioned, it does not change how
parallelization works, but it does accelerate execution in several
ways, one is by turning pipelining always on, others have to do with
optimizing execution for a threaded model and a 'semi persistent'
agent on the target among other things. Before you ask, this cannot be
integrated into the main ansible w/o loosing functionality and the
threading model is not always an improvement over the forks, depends a
lot on the size of the load and how it is structured, it gets very
complicated at on point.

--
----------
Brian Coca