Setting -f from within a playbook

[Yves Dorfsman]

Is there a way to set the number of forks from within a playbook?

The default is fine for most of our playbooks, but one of them benefits from a
higher number. I don't want to rely on people remembering to set -f20 for this
particular playbook (that's why we use ansible, humans are unreliable!).

--
Yves. http://www.SollerS.ca/
Unix/Linux and Python specialist in Calgary.
http://blog.zioup.org/

[Yves Dorfsman]

On 2013-03-13 11:07, Yves Dorfsman wrote:
>
> Is there a way to set the number of forks from within a playbook?
>
> The default is fine for most of our playbooks, but one of them benefits from a
> higher number. I don't want to rely on people remembering to set -f20 for this
> particular playbook (that's why we use ansible, humans are unreliable!).
>

Anyway to do this?

Now I have another case where we install a piece of software that registers
against a manager, but, that manager server can only accept one registration
at a time... Being able to force "-f 1" (forks = 1 ???) would solve this issue
(and the one above!).

Because we don't want to rely on human memory (remembering to use -f1 when
running the playbook, or running the playbook once against each host with -l),
we ended up creating one group per host - clunky!

[benno joy]

Yves,

I think you can use the 'serial:' option here, maybe there's a better way.

serial: 1 means only one host at a time

serial: 30 , 30 forks

Thanks,

Benno

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Michael DeHaan]

Serial does not set the number of forks, it addresses the number of
hosts to manage at one time versus doing the first task across the
top.

I agree it should be made a playbook setting, but it's also
configurable in the configuration file, so this should be sufficient
for most.

--Michael

>> email to ansible-proje...@googlegroups.com.

>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to ansible-proje...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Yves Dorfsman]

On 2013-03-20 11:16, benno joy wrote:
>
> I think you can use the 'serial:' option here, maybe there's a better way.
>
> serial: 1 means only one host at a time
> serial: 30 , 30 forks
>

Ah! I've just done a bunch of tests....

-it works well to limit the number of forks on tasks (serial < 5)

-it never works on gather_facts. gather_facts is always run in parallel, five
hosts at a time regardless of the serial option (unless -f is used)

-it does not work if you set serial > 5, it still runs 5 hosts at once (both
for gather_facts and a task)

[Michael DeHaan]

serial is the construct used to define rolling updates behavior.

and only that.

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to ansible-proje...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Yves Dorfsman]

On 2013-03-20 11:46, Michael DeHaan wrote:
> Serial does not set the number of forks, it addresses the number of
> hosts to manage at one time versus doing the first task across the
> top.
>
> I agree it should be made a playbook setting, but it's also
> configurable in the configuration file, so this should be sufficient
> for most.
>

But aren't the settings in the configuration file applied to all the playbooks?

I'm looking for a set'table fork on a per playbook basis (I only have a few
playbooks where I need a high fork number).

[Michael DeHaan]

I understand.

You may wish to try Brian Coca's trick of putting a shebang line in
your playbook.

I thought that was pretty clever!

#!/usr/bin/ansible-playbook --forks 20

etc

I agree it should be a feature, please open a github ticket so we can track it.

--Michael

[Yves Dorfsman]

On 2013-03-20 11:46, Michael DeHaan wrote:
>

> I agree it should be made a playbook setting, but it's also
> configurable in the configuration file, so this should be sufficient
> for most.
>

Is -f an upper limit or an absolute number of processes started?

If I have a group of host with 3 hosts in it, will ansible-playbook spin up 5
processes anyway (assuming forks=5), or will it only fork what he needs?

I'm trying to figure out the disadvantages of setting a high number of fork
for all playbooks, even the ones that don't necessarily need it.

Thanks.

[Michael DeHaan]

forks just uses multiprocessing.py straight, so it will create 5 if --forks=5.

Given sufficient memory, --forks 50 is not unheard of, but can get a
little RAM hungry at those levels.

Generally folks are almost always setting the serial value at those
levels (rolling update mode) and it does not make sense to have forks
exceed the serial count.

There really isn't a reason we can't auto-minimize the fork count when
it is not needed.

--Michael

[Yuri Lukyanov]

Are there any new considerations on this?

Being able to specify a number of forks in a play would be really helpful.

My use case is that in a playbook I have a play to 'git pull' and some other plays (compile the code, for example).

The thing is that the git server I use has some restrictions on the number of simultaneous connections.

So I'd like to run the first play with a certain number of forks and the rest as much parallel as possible.

The workaround for me would be to use 'serial', but it runs slowly as it waits all the hosts in a series and only then runs the next one.

четверг, 21 марта 2013 г., 0:33:36 UTC+4 пользователь Michael DeHaan написал:

[Michael DeHaan]

Nothing at this time.

To post to this group, send email to ansible...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansible.com>
CTO, Ansible, Inc.
http://www.ansible.com/