venv installing system wide pip packages into venv virtual environment in ansible

[cool prat]

Hi All,

I am running ansible to install some pip packages into virtual environment using `venv` but its installing exact same version and no. packages from system wide python and not what is the mentioned in `requirement.txt`.

I am able to achieve success when i am doing it manually. i.e system wide pip packages and virtual environment pip packages are different. 

I am getting below error in ansible

```

TASK [curator : Install pip packages List.] ****************************************************************************************************************************

fatal: [elasticsearch_1]: FAILED! => {

    "changed": false,

    "cmd": "/home/user/elk-venv/bin/pip list --format=freeze",

    "rc": 13

}

MSG:

[Errno 13] Permission denied

```

Steps i am doing through ansible task are - 

1. Creating `venv` using command module as below 

```

# /usr/bin/python3.6 -m venv /home/user/elk-venv

```

2. Copying `requirements.txt` file and `tranferred_packages` folder inside `/home/user/elk-venv/`.

`requirements.txt` has list of pip packages and `tranferred_packages` folder has downloaded packages. basically installing packages offline way.

3. Using pip module trying to install packages as below,

```

- name: Install pip packages List.

  pip:

    requirements: /home/user/elk-venv/requirements.txt

    extra_args: "--no-index --find-links=file:////home/user/elk-venv/tranferred_packages"

    virtualenv: /home/user/elk-venv

virtualenv_python: /home/user/elk-venv/bin/python3.6

```

Below is the virtual environment created by ansible (elk-env)

Getting permission denied when trying with `bin/pip3 list`.

```

(elk-venv) [root@Server elk-venv]# bin/pip3 list

-bash: bin/pip3: Permission denied

```

able to list the pip3 list like below, but it seems to be copy of system python

```

(elk-venv) [root@Server elk-venv]# pip3 list

attrs (19.3.0)

blist (1.3.6)

```

If we see below it shows `/usr/local/lib/python3.6/site-packages`.

```

(elk-venv) [root@Server elk-venv]# pip3 show pytz

Name: pytz

Version: 2019.3

Summary: World timezone definitions, modern and historical

Home-page: http://pythonhosted.org/pytz

Author: Stuart Bishop

Author-email: stu...@stuartbishop.net

License: MIT

Location: /usr/local/lib/python3.6/site-packages

Requires:

(elk-venv) [root@Server elk-venv]#

```

Not sure why but config file also showing `/usr/bin`

```

(elk-venv) [root@Server elk-venv]# cat  pyvenv.cfg

home = /usr/bin

include-system-site-packages = false

version = 3.6.8

(elk-venv) 

```

pip/python files under bin folder. `python3.6` is pointing to `/usr/bin/python3.6`

```

(elk-venv) [root@Server bin]# ls -lrth

lrwxrwxrwx 1 root root   18 Apr 28 21:56 python3.6 -> /usr/bin/python3.6

lrwxrwxrwx 1 root root    9 Apr 28 21:56 python3 -> python3.6

lrwxrwxrwx 1 root root    9 Apr 28 21:56 python -> python3.6

-rwxr-xr-x 1 root root  224 Apr 28 21:56 pip

-rwxr-xr-x 1 root root  224 Apr 28 21:56 pip3.6

-rwxr-xr-x 1 root root  224 Apr 28 21:56 pip3

```

files under `elk-venv` venv. Can't see `LICENSE` file here which is there in `devenv`.

```

drwxr-x--- 2 root root 4.0K Apr 28 21:56 include

-rw-r----- 1 root root   69 Apr 28 21:56 pyvenv.cfg

lrwxrwxrwx 1 root root    3 Apr 28 21:56 lib64 -> lib

drwxr-x--- 3 root root 4.0K Apr 28 21:56 lib

drwxr-x--- 2 root root 4.0K Apr 28 21:56 bin

-rw-r----- 1 root root 1022 Apr 28 21:56 requirements.txt

drwxr-x--- 3 root root 4.0K Apr 28 21:56 tranferred_packages

```

Created below virtual environment manually (devenv) it has its own pip packages.

here able to list the packages. Not getting permission denied. 

```

(devenv) [root@Server devenv]# bin/pip list

Package               Version

--------------------- ----------

APScheduler           3.6.3

attrs                 19.3.0

pip                   21.1

```

It shows package from its own path i.e `/root/devenv/lib/python3.6/site-packages`.

```

# pip3 show pytz

Name: pytz

Version: 2019.3

Summary: World timezone definitions, modern and historical

Home-page: http://pythonhosted.org/pytz

Author: Stuart Bishop

Author-email: stu...@stuartbishop.net

License: MIT

Location: /root/devenv/lib/python3.6/site-packages

Requires:

Required-by: tzlocal, twilio, APScheduler

```

config file is showing `/bin`.

```

(devenv) [root@Server devenv]# cat pyvenv.cfg

home = /bin

include-system-site-packages = false

version = 3.6.8

(devenv) [root@Server devenv]#

```

pip/python files under bin folder. here `python3.6` is pointing to `/bin/python3.6`.

```

(devenv) [root@Server bin]# ls -lrth

total 148K

lrwxrwxrwx 1 root root   14 Apr 27 17:39 python3.6 -> /bin/python3.6

lrwxrwxrwx 1 root root    9 Apr 27 17:39 python3 -> python3.6

lrwxrwxrwx 1 root root    9 Apr 27 17:39 python -> python3.6

-rwxr-xr-x 1 root root  233 Apr 28 12:29 pip3.6

-rwxr-xr-x 1 root root  233 Apr 28 12:29 pip3

-rwxr-xr-x 1 root root  233 Apr 28 12:29 pip

```

files under `devenv` venv.

```

-rw-r----- 1 root root   65 Apr 27 17:39 pyvenv.cfg

lrwxrwxrwx 1 root root    3 Apr 27 17:39 lib64 -> lib

drwxr-x--- 3 root root 4.0K Apr 27 17:39 lib

drwxr-x--- 2 root root 4.0K Apr 27 17:39 include

-rw-r----- 1 root root  12K Apr 28 12:30 LICENSE

drwxr-x--- 3 root root 4.0K Apr 28 13:30 bin

drwxr-x--- 2 root root 4.0K Apr 28 20:45 tranferred_packages

-rw-r----- 1 root root 1022 Apr 28 20:45 requirements.txt

(devenv) [root@Server devenv]#

```

ansible version is - `ansible 2.9.9` \

ansible python version = 2.7.5

I think something missing in the pip module above in anisble causing this. Can someone please tell what it is?

### Component Name

pip

### Ansible Version

```console

ansible 2.9.9

```

### Configuration

```console

$ ansible-config dump --only-changed

ANSIBLE_PIPELINING(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = True

ANSIBLE_SSH_ARGS(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey

ANSIBLE_SSH_CONTROL_PATH(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = /tmp/ansible-%%h

CACHE_PLUGIN(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = jsonfile

CACHE_PLUGIN_CONNECTION(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = /tmp/facts_cache

CACHE_PLUGIN_TIMEOUT(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = 7200

DEFAULT_FORKS(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = 4

DEFAULT_GATHERING(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = smart

DEFAULT_MODULE_PATH(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = [u'/usr/share/ansible', u'/opt/comp/PS_INSTALL/DEV/ansible-develop/rol

DEFAULT_PRIVATE_KEY_FILE(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = /home/obadm/.ssh/id_rsa

DEFAULT_REMOTE_USER(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = obadm

DEFAULT_STDOUT_CALLBACK(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = debug

DEFAULT_TIMEOUT(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = 720

DEFAULT_VAULT_PASSWORD_FILE(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = /opt/comp/PS_INSTALL/DEV/ansible-develop/vault

DEPRECATION_WARNINGS(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = False

HOST_KEY_CHECKING(/opt/comp/PS_INSTALL/DEV/ansible-develop/ansible.cfg) = False

```

### OS / Environment

RHEL 7.8

### Steps to Reproduce

run the above ansible code to reproduce 

### Expected Results

virtual environment (i.e elk-venv in this case) should have its own pip packages. pip3 show command should show its own path and not system paython path.

### Actual Results

```console

TASK [curator : Install pip packages List.] ****************************************************************************************************************************

fatal: [elasticsearch_1]: FAILED! => {

    "changed": false,

    "cmd": "/home/user/elk-venv/bin/pip list --format=freeze",

    "rc": 13

}

MSG:

[Errno 13] Permission denied

```

Thanks,