Trying to hack a module, but external dependency cannot be found despite being installed

[Simon Wydooghe]

I'm following the Developing Modules guide to get myself started with Ansible hacking (http://docs.ansible.com/ansible/dev_guide/developing_modules.html).

I've sourced hacking/env-setup and now I'm simply trying to use test-module to test an original unedited module.

The module I'm trying to test is 'virt', under extras/cloud/extra. It has a python dependency: libvirt-python. This dependency has been installed and I can open a Python terminal and import 'libvirt' without a problem.

root@9ba2749b80b0:/code# python2

Python 2.7.12 (default, Nov 17 2016, 22:18:11) 

[GCC 4.9.2] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import libvirt

>>> 

However, when using test-module to test 'virt', it seems unable to import 'libvirt':

root@9ba2749b80b0:/code# hacking/test-module -m lib/ansible/modules/extras/cloud/misc/virt.py 

* including generated source, if any, saving to: /root/.ansible_module_generated

* ansiballz module detected; extracted module source to: /root/debug_dir

***********************************

RAW OUTPUT

{"msg": "The `libvirt` module is not importable. Check the requirements.", "failed": true, "invocation": {"module_args": {"xml": null, "state": null, "command": null, "name": null, "uri": "qemu:///system"}}}

***********************************

PARSED OUTPUT

{

    "failed": true, 

    "invocation": {

        "module_args": {

            "command": null, 

            "name": null, 

            "state": null, 

            "uri": "qemu:///system", 

            "xml": null

        }

    }, 

    "msg": "The `libvirt` module is not importable. Check the requirements."

}

I must be missing something here... Does anyone have a hint as to what could be the problem?

[Brian Coca]

My guess is that it is probably cause it is running 'python' and not 'python2', you can try running the setup module which includes the python executable information.

----------

Brian Coca

[Greg Langford]

Hello,

You could also try and run the command without quotes 'which python' that will give you the path to the Python executable

Then run again without the quotes '/usr/bin/python --version' replacing the path with the one returned from the which command you ran before which will give you the python version.

Ansible afaik requires Python 2.7 so if the version of the system python is Python 3 you may want to symlink /usr/bin/python to the python2 executable.

Kind Regards

[Simon Wydooghe]

Hey Brian and Greg,

Thanks for the feedback. I run Arch Linux, where python3 is the default python. For this reason, I've tried getting the Ansible hacking setup both in a python2 virtualenv and in a python2 docker image. Both result in the same issue.

This is the output of which python and env python in my Docker image:

root@939639e10ec5:/code# which python
/usr/local/bin/python
root@939639e10ec5:/code# python

Python 2.7.12 (default, Nov 17 2016, 22:18:11)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> exit()
root@939639e10ec5:/code# env python

Python 2.7.12 (default, Nov 17 2016, 22:18:11)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> exit()
root@939639e10ec5:/code#

As you can see, python points to the right python executable. I'm using the official python2 Docker image and have installed the necessary dependencies. This is the Dockerfile:

FROM python:2

RUN pip install paramiko PyYAML Jinja2 httplib2 six

RUN apt-get update && apt-get install -y libvirt-dev \

 && pip install libvirt-python

RUN pip install boto boto3

VOLUME /code

WORKDIR /code

VOLUME /data

CMD ["bash"]

I then load then Ansible repo under /data.

I ran the setup module and it confirms the python executable being 2.7.12:

root@c52acee259c9:/code# ansible localhost -m setup -a filter=ansible_env

 [WARNING]: Host file not found: /etc/ansible/hosts

 [WARNING]: provided hosts list is empty, only localhost is available

localhost | SUCCESS => {

    "ansible_facts": {

        "ansible_env": {

            "ANSIBLE_HOME": "/code", 

            "GPG_KEY": "C01E1CAD5EA2C4F0B8E3571504C367C218ADD4FF", 

            "HOME": "/root", 

            "HOSTNAME": "c52acee259c9", 

            "LANG": "C.UTF-8", 

            "MANPATH": "/code/docs/man:", 

            "PATH": "/code/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 

            "PWD": "/code", 

            "PYTHONPATH": "/code/lib:", 

            "PYTHON_PIP_VERSION": "9.0.1", 

            "PYTHON_VERSION": "2.7.12", 

            "SHLVL": "1", 

            "TERM": "xterm", 

            "_": "/code/bin/ansible"

        }

    }, 

    "changed": false

}

Any more ideas? :)

Regards,

Simon

[Simon Wydooghe]

So, I found the problem. Actually, I'm not sure if it's a problem or desired behaviour :)

When I run 'source hacking/env-setup', the PYTHONPATH environmental variable gets set to the 'lib' directory in the ansible repository used for development. This directory however does not contain the external Python dependencies (installed with pip) which are installed in /usr/lib/python2.7/site-packages. If a PYTHONPATH environmental variable is defined before running 'source hacking/env-setup' however, this original path is prepended to the new PYTHONPATH. So, I'm now simply exporting PYTHONPATH=/usr/lib/python2.7/site-packages before I run env-setup and all is well. I can now use test-module to test modules that rely on external dependencies.

Hopefully this is useful to somebody in a similar situation. Feel free to let me know if I'm overlooking something and made this more difficult than necessary.

Regards,

Simon