How do i optimize performance of Ansible playbook with regards to ssh connections

[Mohtashim S]

I have the below playbook test1.yml that gets istat data for 26 subfolders under this directory /var/myfile/pdf.

tasks:

- name: List directories

  raw: "ls -d "/var/myfile/pdf/*/"

  register: subdir

 - name: List pid files

   raw: "istat {{ item }}"

   with_items: "{{ subdir.stdout_lines }}"

I run the playbook and it takes 29 seconds to complete

time ANSIBLE_SSH_PIPELINING=True ansible-playbook -i=10.9.9.12, -f 30 test1.yml -vvv

After the playbook completes below is the time taken details output:

Output:

real 0m29.144s user 0m6.206s sys 0m5.618s

I now put the same code with istat task in include_tasks` file like below.

Playbook test2.yml

tasks:

- name: List directories

  raw: "ls -d "/var/myfile/pdf/*/"

  register: subdir

 - name: List pid files

   include_tasks: "innertest.yml"

   with_items: "{{ subdir.stdout_lines }}"

cat innertest.yml

   - raw: "istat {{ item }}" time ANSIBLE_SSH_PIPELINING=True ansible-playbook -i=10.9.9.12, -f 30 test2.yml -vvv

Output:

real 0m59.044s user 0m18.203s sys 0m10.118s

As you can see the time with the same amount of task has more than doubled due to include_tasks

In the debug, I also see there are 26 ssh connections triggered for the 26 sub-directories with_items for the same target host 10.9.9.12

I'm not sure of how this works internally but it would have been nice to have a single ssh connection for istat for 26 sub-directories on the same host for performance reasons.

Is there a way to increase the performance for include_tasks and bringing down the number of ssh connections to the same host ?

[Stefan Hornburg (Racke)]

On 10/2/20 3:24 PM, Mohtashim S wrote:
> I have the below playbook test1.yml that gets istat data for 26 subfolders under this directory /var/myfile/pdf.
>
> tasks:
>
> - name: List directories
>   raw: "ls -d "/var/myfile/pdf/*/"
>   register: subdir
>
>  - name: List pid files
>    raw: "istat {{ item }}"
>    with_items: "{{ subdir.stdout_lines }}"
>

What kind of server / device is that? I would rather use "find" module instead of "raw".
What does istat actually?

Regards
Racke

> --
> You received this message because you are subscribed to the Google Groups "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> ansible-proje...@googlegroups.com <mailto:ansible-proje...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/9d1d78fa-d06c-42ec-8aec-f54bc5845506n%40googlegroups.com
> <https://groups.google.com/d/msgid/ansible-project/9d1d78fa-d06c-42ec-8aec-f54bc5845506n%40googlegroups.com?utm_medium=email&utm_source=footer>.


--
Ecommerce and Linux consulting + Perl and web application programming.
Debian and Sympa administration. Provisioning with Ansible.