How to install python packages for existing AWX containers?

1,836 views
Skip to first unread message

Ankit Vashistha

unread,
Sep 20, 2018, 2:48:39 PM9/20/18
to AWX Project
I am trying to use ansible snow module and noticed that the requirement for that is to install pysnow. I have tried installing this by making changes in the existing repo but this always results in error. How can i make this work?

Following are the details.

Docker Containers
# docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                                                 NAMES
80ab6bf562a9        ansible/awx_task:latest      "/tini -- /bin/sh -c…"   2 weeks ago         Up 2 days           8052/tcp                                              awx_task
aea2551951d5        b7c261b76010                 "/tini -- /bin/sh -c…"   2 weeks ago         Up 2 days           0.0.0.0:80->8052/tcp                                  awx_web
e789a4a82a9e        memcached:alpine             "docker-entrypoint.s…"   2 weeks ago         Up 3 days           11211/tcp                                             memcached
a8c74584255c        ansible/awx_rabbitmq:3.7.4   "docker-entrypoint.s…"   2 weeks ago         Up 3 days           4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp   rabbitmq
25f6f6ca7766        postgres:9.6                 "docker-entrypoint.s…"   2 weeks ago         Up 3 days           5432/tcp                                              postgres

Docker Images
# docker images
REPOSITORY             TAG                 IMAGE ID            CREATED             SIZE
postgres               9.6                 106bdfb062df        7 weeks ago         235MB
ansible/awx_task       latest              535bb2b8e1f3        8 weeks ago         1.07GB
ansible/awx_web        <none>              b7c261b76010        8 weeks ago         1.04GB
hello-world            latest              2cb0d9787c4d        2 months ago        1.85kB
memcached              alpine              b40e8fa7e3e5        2 months ago        8.69MB
ansible/awx_rabbitmq   3.7.4               e08fe791079e        5 months ago        85.6MB
[root@clab2726vw0001 ~]#


# docker run -ti ansible/awx_task:latest bash

bash-4.2$ pip install pysnow
The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pysnow
Collecting requests (from pysnow)
    100% |################################| 92kB 1.8MB/s
Collecting oauthlib (from pysnow)
    100% |################################| 122kB 2.8MB/s
Collecting requests_oauthlib (from pysnow)
Collecting httpretty (from pysnow)
    100% |################################| 4.0MB 175kB/s
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/site-packages (from pysnow)
Collecting ijson (from pysnow)
Collecting pytz (from pysnow)
    100% |################################| 512kB 1.4MB/s
Collecting idna<2.8,>=2.5 (from requests->pysnow)
    100% |################################| 61kB 3.9MB/s
Collecting chardet<3.1.0,>=3.0.2 (from requests->pysnow)
    100% |################################| 143kB 3.6MB/s
Collecting urllib3<1.24,>=1.21.1 (from requests->pysnow)
    100% |################################| 143kB 3.7MB/s
Collecting certifi>=2017.4.17 (from requests->pysnow)
    100% |################################| 153kB 3.5MB/s
Installing collected packages: idna, chardet, urllib3, certifi, requests, oauthlib, requests-oauthlib, httpretty, ijson, pytz, pysnow
  Found existing installation: idna 2.4
    Uninstalling idna-2.4:
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 326, in run
    strip_file_prefix=options.strip_file_prefix,
  File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 736, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/lib/python2.7/site-packages/pip/req/req_install.py", line 742, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/usr/lib/python2.7/site-packages/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/usr/lib64/python2.7/shutil.py", line 299, in move
    rmtree(src)
  File "/usr/lib64/python2.7/shutil.py", line 252, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/lib64/python2.7/shutil.py", line 250, in rmtree
    os.remove(fullname)
OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/idna/idnadata.pyo'
You are using pip version 8.1.2, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.


bash-4.2$ pip install --upgrade pip
The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
    100% |################################| 1.3MB 684kB/s
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Uninstalling pip-8.1.2:
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 326, in run
    strip_file_prefix=options.strip_file_prefix,
  File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 736, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/lib/python2.7/site-packages/pip/req/req_install.py", line 742, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/usr/lib/python2.7/site-packages/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/usr/lib64/python2.7/shutil.py", line 302, in move
    os.unlink(src)
OSError: [Errno 13] Permission denied: '/usr/bin/pip'
You are using pip version 8.1.2, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Christoph Wegener

unread,
Sep 21, 2018, 8:05:33 AM9/21/18
to Ankit Vashistha, AWX Project
On Fri, 21 Sep 2018 at 04:48, Ankit Vashistha wrote:
> I am trying to use ansible snow module and noticed that the
> requirement for that is to install pysnow. I have tried installing
> this by making changes in the existing repo but this always results in
> error. How can i make this work?
>
>
> *# docker run -ti ansible/awx_task:latest bash*
>
> *bash-4.2$ pip install pysnow*

AWX makes use of Python virtualenv for playbook execution. Your commands
were trying to install the 'pysnow' package into the global python scope
of the running docker image, which is not the recommended way to install
the requirements.

The virtualenv directory used for executing playbooks is located at
`/var/lib/awx/venv/ansible`.

So, in order to manually install a python package using pip into the
virtulenv, you would run the following commands inside the 'awx_task'
container:

source /var/lib/awx/venv/ansible/bin/activate

pip install pysnow

Alternatively, you can simple create playbook that uses the `pip` module
which you can then run as a job in AWX in order to ensure that the
required python modules are present in the virtualenv. This method is
much preferred over the manual way of installing python packages.

Ankit Vashistha

unread,
Sep 21, 2018, 8:24:40 AM9/21/18
to Christoph Wegener, AWX Project
Thanks Chris,

I was able to create virtual environment and install the packages inside that in aws_tasks container. Post this I have done the commit and it created a new image. How can I use this new image to work with the container with changes. I am having hard time in making the changes work. 

Ankit Vashistha

unread,
Sep 24, 2018, 4:52:16 AM9/24/18
to Christoph Wegener, AWX Project
Hi All,

Can anyone please help in this case?
Currently my docker containers are as follows.

# docker container ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                    PORTS                                                 NAMES
968fb2a7da2f        5290f9b3268c                 "/tini -- /bin/sh -c…"   2 days ago          Exited (143) 2 days ago                                                         awx_task
80ab6bf562a9        535bb2b8e1f3                 "/tini -- /bin/sh -c…"   3 weeks ago         Up 2 days                 8052/tcp                                              awx_task_OLD
aea2551951d5        b7c261b76010                 "/tini -- /bin/sh -c…"   3 weeks ago         Up 2 days                 0.0.0.0:80->8052/tcp                                  awx_web
e789a4a82a9e        memcached:alpine             "docker-entrypoint.s…"   3 weeks ago         Up 2 days                 11211/tcp                                             memcached
a8c74584255c        ansible/awx_rabbitmq:3.7.4   "docker-entrypoint.s…"   3 weeks ago         Up 2 days                 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp   rabbitmq
25f6f6ca7766        postgres:9.6                 "docker-entrypoint.s…"   3 weeks ago         Up 2 days                 5432/tcp                                              postgres

Here, when i enable latest container (968fb2a7da2f) with latest image (with changes), it doesn't work. It appears like, the new container is not able to talk to other AWX containers. When i start the old container (80ab6bf562a9), it works normally. In the above snapshot, i have enabled the older container and disabled the new one.

I am stuck at this point and not able to move forward, any help would be really appreciated.

Thanks and Regards,

Ankit Kumar Sharma
Mobile +91-9717144596

Reply all
Reply to author
Forward
0 new messages