Seeking testers for a (radically) performance-improving Ansible plug-in

[David Wilson]

Hi guys,

I've been working occasionally on a new connection method for Ansible that drastically improves its efficiency, even when compared to pipelining and SSH multiplexing. The typical speedups can be impressive - in the 1.5x to 5x range, and some nice side effects exist such as system logs not being spammed while Ansible executes.

    http://mitogen.readthedocs.io/en/latest/ansible.html

It takes all of 5 minutes over lunch to try out, and I'd love to get feedback on it from a variety of playbooks, if in no other form than bug reports. ;)

Thanks,


David

[dawg...@googlemail.com]

Very interesting, just brought up https://github.com/ansible/ansible/issues/36275.

Mitogen (or basically any short-lived agent) seems like a feasible approach to solve the problem, albeit sending python modules via ssh and executing them in a long(er)-running process adds quite some complexity.

What's wrong with just caching python modules (or binaries) in a size-limited temp folder? Using content-addressing should get rid of possible versions conflicts and a tiny bit of disk or tmpfs space doesn't seems like an unreasonable requirement.

https://github.com/dw/mitogen/issues/85 :)

[David Wilson]

It's certainly possible to cache modules, the problem is that aside from requiring a writeable disk, it introduces intermediate persistent state, which is evil in any software design where it can otherwise be avoided, as it creates all kinds of coherency and management issues (of which you name one - limiting the size!), and greatly increases the number of generally difficult/impossible to test states the software may find itself in.

I'm not averse to adding it, but meanwhile there are more ripe performance hindrances to address beforehand.

[Brian Coca]

So during vacation i was toying with an 'ansible remote execution'
(rme) https://github.com/bcoca/ansible/tree/rme , via a connection
plugin that requires Ansible be installed on the target node.

It could either have a service/socket/trigger to run ansible tasks as
passed in or we can just execute `ansible -m module ...` directly.

This avoids 'cache invalidation issues' but does introduce version
mismatch ones as well as 'agent or required software' on the targets.
More thorny issues are 'what do we pass', templated task? task loop?
original task + vars? what secrets do we pass and how?

I have not seen an approach that does not have drawbacks, in the end
we might implement more than one and let the user choose their
tradeoffs.


----------
Brian Coca