High memory usage when running multiple ansible playbooks on localhost

[pradeep.hk]

I have a requirement to run lots of playbooks parallely on localhost. 

When I run 100 playbooks parallely (a simple one that executes the a shell command - date) , I see that

it consumes lot of resources (memory ~ 7GiB and CPU ~ 200%)

Is this on expected lines ?

here is what I am trying to run

--------------------main.yml-----------------------------------------

- hosts: 127.0.0.1

  connection: local

  gather_facts: no

 

  tasks:

    - name: Get date

      shell: date

      register: result

     

    - debug:

        var: result.stdout

-------------------------------------------------------------

I execute multiple of them with the following script

-------------------------------------------------------------

#!/bin/bash

COUNT=1

if [ $# -ne 0 ]

then

   COUNT=$1

fi

for ((i=1;i<=$COUNT;i++)); 

do 

   ansible-playbook main.yml &

done

-------------------------------------------------------------

[Karl Auer]

what happens if you leave Ansible out and just fire off 100 copies of date?

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/fb716bff-d851-4f21-8542-71241b925940%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Karl Auer

Email  : ka...@2pisoftware.com
Website: http://2pisoftware.com

GPG/PGP : 958A 2647 6C44 D376 3D63 86A5 FFB2 20BC 0257 5816
Previous: F0AB 6C70 A49D 1927 6E05 81E7 AD95 268F 2AB6 40EA

[pradeep.hk]

When I fire off 100 copies of date, I hardly see any change in memory usage.

I even fired 100 copies of the following script, but I donot see any change in memory usage even when 100 such processes are running in parallel

-------------------------------------------------------

#!/bin/bash

COUNT=1000000

for ((i=1;i<=$COUNT;i++)); 

do    

  :

done

-------------------------------------------------------

[Karl Auer]

That script does not run 100 copies of date. It runs 100 copies of an empty loop which may well be optimised out of existence.

I'm not saying that Ansible is not the problem, but you do need to start with a fair comparison. Also, your playbook is running 100 times on one system; you may find it is hardly a problem if you run those 100 scripts on 100 remote systems.

Regards, K.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/132ce039-63c5-4f0d-9ab6-051a0d630a26%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Kai Stian Olstad]

On 12.12.2018 07:53, pradeep.hk wrote:
> I have a requirement to run lots of playbooks parallely on localhost.
> When I run 100 playbooks parallely (a simple one that executes the a
> shell
> command - date) , I see that
> it consumes lot of resources (memory ~ 7GiB and CPU ~ 200%)
> Is this on expected lines ?

It sound about right.

For small execution Ansible uses about 50MB of memmory, 50MB * 100 is
about 5GB.


But since you are running the same playbook on the same host you could
instead do this

In inventory add

[testgroup]
testhost[001:100

And in ansible.cfg set fork=100

And the playbook

- hosts: testgruop

connection: local
gather_facts: no
tasks:
- name: Get date
shell: date
register: result

- debug:
var: result.stdout

This will only use a little over 50MB and not GB of memory and a lot
less CPU.


Your test case seams to me not very relevant, but if it is, let Ansible
fork and not the OS.

--
Kai Stian Olstad

[pradeep.hk]

Few things:

(1)As mentioned in my previous reply, I even ran the date command 100 times but no change in memory usage. In addition, I also tried a empty loop to keep the process running

(2)It is my requirement to run the playbook on the localhost in response to a request to execute a job (there could be multiple such requests)

[pradeep.hk]

The requirement is to be able to launch a playbook on receiving a request (ie on demand). So, if 100 requests are received, it will result in 100 playbooks being executed parallely.

You mentioned - Ansible uses about 50MB of memory per playbook. Is there something that can be done to optimize on that ?

[Kai Stian Olstad]

On 12.12.2018 11:22, pradeep.hk wrote:
> The requirement is to be able to launch a playbook on receiving a
> request
> (ie on demand). So, if 100 requests are received, it will result in 100
> playbooks being executed parallely.
> You mentioned - Ansible uses about 50MB of memory per playbook. Is
> there
> something that can be done to optimize on that ?

50MB is just the minimum, it might be a lot more depending on playbook.
I highly doubt there is much that can be done.

Just a Hello World in python uses 7MB

$ /usr/bin/time -f %M python -c 'print "Hello World"'
Hello World
Mem used in kB: 7316



An Ansible equivalent Hello World uses 48MB

$ /usr/bin/time -f "Mem used in kB: %M" ansible-playbook test.yml

PLAY [localhost]
***********************************************************

TASK [debug]
***************************************************************
ok: [localhost] => {}

MSG:

Hello World


PLAY RECAP
*****************************************************************
localhost : ok=1 changed=0 unreachable=0
failed=0

Mem used in kB: 48316


If you use Ansible dynamic inventory you can easily change the inventory
depending on the dynamic demand.

--
Kai Stian Olstad

[Kai Stian Olstad]

On Wednesday, 12 December 2018 12:32:30 CET Kai Stian Olstad wrote:
> If you use Ansible dynamic inventory you can easily change the inventory
> depending on the dynamic demand.

You don't even need to use dynamic inventory, this will do the same as your script but let ansible do the forking.

#!/bin/bash
ansible-playbook -i $(seq -s '' -f '%03g,' ${1-1}) -f ${1-1} main.yml


And your your playbook must change to "hosts: all"

--
Kai Stian Olstad

[pradeep.hk]

Got it. Thanks for your time.

My requirement is to be able to sequence together tasks(to create a workflow) and after some prototyping, I found ansible playbook to meet that requirement.

But then, scalability is turning out to be an issue. If we could daemonize the ansible process so that it can receive requests and execute playbook in a thread, that would scale.

But then, that is not how ansible is supposed to work

[Jonathan Lozada De La Matta]

tou ahould check out AWX

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6d1609ed-7b2e-4067-af41-ad40de8e78ff%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jonathan lozada de la matta

AUTOMATION PRACTICE