Ansible Without Python?

[Jon Forrest]

I had a radical thought the other day. Would it be possible to
replace Python as the language used by Ansible with a compiled language,
perhaps Go because of its fast compile and link speed? Note that I'm not
talking about something like the existing Ansible shell module, which is
only useful in environments without Python (or Go).

This is actually two questions because the answer depends on
whether we're talking about the controller or the client.
On the controller it would be a simple matter of software
to rewrite Ansible to produce the same Python it produces
today. I suspect, but have no data to support, that an Ansible
controller would run much faster this way, especially in large
environments.

Go Ansible on the client would be much harder and I think is less
likely to produce positive results. Obviously each client
would have to have a working Go environment. I suspect that
it would be no more difficult to make this happen than it is
to put a working Python environment on a client.

At first glance it isn't clear that the size of such an environment
would be smaller in Go than in Python. Indeed, the size of the Go 1.16
installer for Windows is roughly 100MB larger than that for
Python 3.9.2. However, if this is an issue I suspect that a striped down
working Go environment could be created that's much smaller.

Another step necessary on the client would be the creation of
a pre-compiled Go Ansible runtime. With this present on each
client, the Ansible controller would only have to send the
minimal amount of Go code to the client, which would then compile
that code, and link it with the Go Ansible runtime. It's only
because Go has such an incredibly fast compiler and linker that
Go Ansible is worth considering.

I suspect, but again can't prove, that various clever caching techniques
could be using on the client to reduce the amount of compiling
and linking that needs to be done.

For simple small environments what I'm thinking about wouldn't make much
difference. However, there are some huge Ansible environments out there
that I could see benefiting from what I have in mind.

Thoughts?

Cordially,
Jon Forrest

[Wei-Yen Tan]

Technically I think you could run ansible in any language. But there would have to be a deep rethink on how to reproduce what had already been created in the python scripts that make up ansible 'workflow'. there is a lot of moving parts in the backend that we don't see. 

Get Outlook for iOS

---

From: ansible...@googlegroups.com <ansible...@googlegroups.com> on behalf of Jon Forrest <nob...@gmail.com>
Sent: Thursday, February 25, 2021 8:33:16 AM
To: ansible...@googlegroups.com <ansible...@googlegroups.com>
Subject: [ansible-project] Ansible Without Python?

 

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/s169lu%24nbn%241%40ciao.gmane.io.

[Jean-Yves LENHOF]

Hi,

There's part of the code that run on the controller, but also on the client.

Today you can run the same ansible code on unix and linux server because
the python binary is the one used on the client side, if using a
compiled language you need to have all the binary version of all your
potentials clients os.

Regards,

JYL

[Wei-Yen Tan]

Well on the client side you can run any language. Ansible uses the shebang on the task module to execute the script. I have even written a custom module that runs powershell on a linux client. You could run bash. As long as you output json object  ansible will just execute and return it back. 

Get Outlook for iOS

---

From: ansible...@googlegroups.com <ansible...@googlegroups.com> on behalf of Jean-Yves LENHOF <jean...@lenhof.eu.org>
Sent: Thursday, February 25, 2021 9:16:57 AM
To: ansible...@googlegroups.com <ansible...@googlegroups.com>
Subject: Re: [ansible-project] Ansible Without Python?

 

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/bdcd2006-79e7-9155-8007-2678dd3166fd%40lenhof.eu.org.

[Jean-Yves LENHOF]

Le 24/02/2021 à 21:21, Wei-Yen Tan a écrit :

Well on the client side you can run any language. Ansible uses the shebang on the task module to execute the script. I have even written a custom module that runs powershell on a linux client. You could run bash. As long as you output json object  ansible will just execute and return it back.

Good luck on porting all the module python code with shell code...

The idea is using a script and not a compiled program is to have not to install something on the client side... If you want a binary on the client side, you're just doing another sort of puppet like language with a necessary client !

The main target of ansible is linux and python is a prerequirements to follow the LSB standard, so it is always installed on any linux !

Regards,

JYL

[Sam Doran]

Jimi actually did this a while ago to see if there were any benefits. In the end, there was no real benefit to moving to Go since most of the time the control node is waiting for code to execute and return results from the managed nodes. There are probably other details I'm forgetting since this was a while ago. I'll let Jimi comment further.

It's possible to use Go modules with Ansible today since we support binary modules. But the problems with that were already outlined.

---

Sam

Ansible Core

[Wei-Yen Tan]

I think we are talking cross purposes here. You can create a 'library' as in task. run in any language. I will have a check to see how it executes. But I am pretty sure it executes as bash With  file  you give it. 

Get Outlook for iOS

---

From: ansible...@googlegroups.com <ansible...@googlegroups.com> on behalf of Jean-Yves LENHOF <jean...@lenhof.eu.org>

Sent: Thursday, February 25, 2021 9:38:02 AM

To: ansible...@googlegroups.com <ansible...@googlegroups.com>
Subject: Re: [ansible-project] Ansible Without Python?

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/63fb6c9a-d42d-0eec-10bc-099899f378fc%40lenhof.eu.org.

[Jon Forrest]

On 2/24/2021 12:42 PM, Sam Doran wrote:
> Jimi actually did this a while ago <https://github.com/jimi-c/ansigo> to

> see if there were any benefits.

I wasn't aware of this. Thanks for the reference.

> In the end, there was no real benefit to
> moving to Go since most of the time the control node is waiting for code
> to execute and return results from the managed nodes.

I'd like to learn how complicated the playbooks and host files were that
he used for the experiment. As I said, in small environments I wouldn't
expect much of an improvement. But, I once worked at a place that had
~1,000 hosts and a very complicated group structure where I think a
compiled language would have made a difference on the controller.

>T here are probably

> other details I'm forgetting since this was a while ago. I'll let Jimi
> comment further.

I'm looking forward to this.

One other thing that's true now is the the Go linker has gotten ~30%
faster.

> It's possible to use Go modules with Ansible today since we support
> binary modules. But the problems with that were already outlined.

I hadn't known about binary modules (it's been a while since I used
Ansible). That's an interesting idea. Given Go's ability as a cross
compiler, I could see the controller compiling and linking modules
in the proper format for a particular client. I suspect, though,
that binary modules would be much larger than the amount of Go source
code that would have to be transferred. Whether that's important is
another question.

Thanks for all the responses.
Jon

[Jean-Yves LENHOF]

Le 24/02/2021 à 22:18, Jon Forrest a écrit :
> On 2/24/2021 12:42 PM, Sam Doran wrote:
>> Jimi actually did this a while ago <https://github.com/jimi-c/ansigo>
>> to see if there were any benefits.
>
> I wasn't aware of this. Thanks for the reference.
>
>> In the end, there was no real benefit to moving to Go since most of
>> the time the control node is waiting for code to execute and return
>> results from the managed nodes.
>
> I'd like to learn how complicated the playbooks and host files were that
> he used for the experiment. As I said, in small environments I wouldn't
> expect much of an improvement. But, I once worked at a place that had
> ~1,000 hosts and a very complicated group structure where I think a
> compiled language would have made a difference on the controller.

First things to try are :

- Playing with the number of forks on the ansible config
(https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.html#setting-the-number-of-forks)


Another way to speed up but with some CAVEATS is to use some material of
mitogen

https://mitogen.networkgenomics.com/ansible_detailed.html


Regards,

JYL

[Brian Coca]

As for having a compiled controller, you must also find a way to
support the templating language (jinja2, a python lib) all the other
plugin types which are python and run in the controller memory:
inventory, action, terminal, shell, become, filter, tests, lookups,
etc.



--
----------
Brian Coca