Preserve virtual env for python when calling third party modules

[Eric Salomé]

Hi,

I am writing third party modules in Python using the tag 

"#<<INCLUDE_ANSIBLE_MODULE_COMMON>>" 

so that I get parameters for my modules in a nice ANSIBLE_MODULE_ARG= {... } json list, read by a call to AnsibleModule().

However, I am using a virtual env for python : /srv/ansible/venv/bin/python

Actually, I need to set in "vars" of all my playbooks, the following variable :

    vars:

       ansible_python_interpreter: /srv/ansible/venv/bin/python

Or (even better) in the [all:vars] section of my inventory

   [all:vars] 

       ansible_python_interpreter=/srv/ansible/venv/bin/python

If I don't do it,  lib/ansible/executor/module_common.py is handling my module in a way that leads to using /usr/bin/python by default

(which is obviously not what I want)

Would a PR useful to tell module_common.py to use "sys.executable" (the current python executable path), instead of "/usr/bin/python" as the "default" python executable path ?

Will this PR be acceptable :

--- module_common.py.orig       2017-04-14 14:10:25.496689174 +0200

+++ module_common.py    2017-04-29 19:58:40.326331537 +0200

@@ -30,6 +30,7 @@

import zipfile

import random

import re

+import sys

from io import BytesIO

 

from ansible.release import __version__, __author__

@@ -709,9 +710,9 @@

                     ' Look at traceback for that process for debugging information.')

         zipdata = to_text(zipdata, errors='surrogate_or_strict')

 

-        shebang, interpreter = _get_shebang(u'/usr/bin/python', task_vars)

+        shebang, interpreter = _get_shebang(sys.executable, task_vars)

         if shebang is None:

-            shebang = u'#!/usr/bin/python'

+            shebang = u'#!{0}'.format(sys.executable)

 

         # Enclose the parts of the interpreter in quotes because we're

         # substituting it into the template as a Python string

-----------------

Best regards - Eric

[Matt Martz]

Ansible will already utilize sys.executable for localhost, as long as you do not explicitly define localhost in your inventory, and instead allow ansible to create it's implicit localhost.  See:

https://github.com/ansible/ansible/blob/0d5d5f2bf66b5e7519269a454ccc3c4b3196d91c/lib/ansible/inventory/__init__.py#L520-L540

Telling ansible to use sys.executable as the default would likely not be what anyone wants. Since the local sys.executable, is likely not the same path as the remote python binary.  /usr/bin/python is predictable for remote target systems, allowing to be overridden by ansible_python_interpreter.

--
You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Martz
@sivel
sivel.net

[Eric Salomé]

Hi,

The example I mentioned was about launching modules on the localhost system.
I don't think "module_common.py" was meant to launch modules on remote target systems, is it ?

I might be mistaking. Can anyone help us on this subject ?


-- Best regards, Eric

[Matt Martz]

module_common.py is used to build out how the module will be executed, regardless of local or remote.

My answer about utilizing implicit localhost is the way to go.

--

You received this message because you are subscribed to the Google Groups "Ansible Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Eric Salomé]

Hi Matt,

Thank you for your concern. I understand (I hope better now) that "module_common" is run to build modules  that will be launched either on local or on remote system.

When building for a local execution of the module, it should take care to search for the 'implicit' localhost "ansible_python_interpreter", or look for sys.executor.
When building for a remote execution of the module, it's ok to call "_get_shebang()" and get the ansible_python_interpreter of the remote system from task_vars.

My playbook is run with a list of remote hosts each possibly having different "ansible_python_interpreter".

I am stuck with that problem, since my remote systems don't have the same python interpreter path as the local one running ansible and ... my modules are run on my local system when "playing" the remote systems.
Any help wanted :-)

[Matt Martz]

We already have functionality to support what you want.

I think you need to move this over to ansible-project, share your playbook, and ask for help on how to get what you want.

Again, use implicit localhost, don't define localhost in your inventory.  Use delegate_to: localhost, or local_action, or target localhost in your hosts specification to use implicit localhost.

Don't use `connection: local`

--
You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eric Salomé]

Hi,

Again, thank you for your time and concern.

What I intended to do is very simple : I need to write a module similar "ios_command", launched in a very simple playbook  :

The module (ios_command or my own module) runs on the localhost (thus needs to use my dedicated environment for python /srv/ansible/venv/bin/python), then connects to the target host to do things there.

1. Normal play
- the ios_command runs well, my own module is called locally with /usr/bin/python, runs out of the dedicated environment (thus do not find its entry point and abort)

2. Play with ansible_python_interpreter=/srv/ansible/venv/bin/python in inventory for each target host (which is not what I should do, but then)
- the ios_command runs well, and my own module also

3. Normal play - my own module includes the "WANT_JSON" tag
- the ios_command runs well, and my own module also

The tag "# WANT_JSON" mainly breaks the way ansible-play will send the parameters to my own module (but I can handle that), and *as a side effect*, tells module_common.py to treat my own module as it is without trying to ANSIBALLZ it or rewrite the shebang.

For the time being, I will use either 3. or 2. .... but I guess there is a better way to handle the problem (since "native" ansible module are working well).
This is a "how to write a module, so that it is launched with the correct shebang" issue

-- Best regards, Eric
 

[Eric Salomé]

Hi Matt,

In the meantime, I studied your proposals and the thing about "modules are run on the remote targets" :

I thought "modules" were run on the local controller by default :

      Modules are run on the remote targets by default using "Normal action plugin".
      What I need is "Another action plugin" to run my modules on the local ansible controller.
      I will check in detail how ios_command do the job and see how to do the same

Thanks for your help.
-- Best regards, Eric

[Eric Salomé]

By the way,

For those interessed in that subject, I discovered (latelly, sorry for that) a full discussion of it in https://github.com/ansible/ansible/issues/16724

Looks like using "local_action: module" (as suggested by Matt) is the most reasonable way to do what I wanted.

-- Best regards, Eric