Pip module issue - install seems to hang when installing from a requirements file

2,393 views
Skip to first unread message

Jason brooks

unread,
Aug 13, 2013, 11:32:54 PM8/13/13
to ansible...@googlegroups.com
Can anyone help with this issue...

When log into the remote machine and manually pip install -r production.txt it works fine 

In my playbook i have:
  - name: Update the virtualenv with production requirements.
    pip: requirements=${project_root}/myapp/deps/production.txt virtualenv=${project_root}/env extra_args='--exists-action w'

Also Tried:
  - name: pip install prod requirements
    command: "${project_root}/env/bin/pip install -r ${project_root}/myapp/deps/production.txt --exists-action w --log ${project_root}/downloads/mypip_log.txt"

ansible 1.2.2
python 2.7.4
pip 1.4.1
remote machine - Ubuntu 13.04

ansible log:

TASK: [Upgrade the virtualenv with prod requirements.] ************************ 
<198.58.115.201> ESTABLISH CONNECTION FOR USER: root
<198.58.115.201> EXEC ['sshpass', '-d9', 'ssh', '-tt', '-q', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/tmp/ansible-ssh-%h-%p-%r', '-o', 'Port=22', '-o', 'GSSAPIAuthentication=no', '-o', 'PubkeyAuthentication=no', '-o', 'User=root', '-o', 'ConnectTimeout=999999', '198.58.115.201', "/bin/sh -c 'mkdir -p /tmp/ansible-1376450420.48-23411754031827 && chmod a+rx /tmp/ansible-1376450420.48-23411754031827 && echo /tmp/ansible-1376450420.48-23411754031827'"]
<198.58.115.201> REMOTE_MODULE pip requirements=/home/myuser/projects/myapp/deps/production.txt virtualenv=/home/myuser/projects/env extra_args='--exists-action w --log ~/pip_prod1_log.txt'
<198.58.115.201> PUT /tmp/tmpShRoOh TO /tmp/ansible-1376450420.48-23411754031827/pip
<198.58.115.201> EXEC ['sshpass', '-d9', 'ssh', '-tt', '-q', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/tmp/ansible-ssh-%h-%p-%r', '-o', 'Port=22', '-o', 'GSSAPIAuthentication=no', '-o', 'PubkeyAuthentication=no', '-o', 'User=root', '-o', 'ConnectTimeout=999999', '198.58.115.201', "/bin/sh -c 'chmod a+r /tmp/ansible-1376450420.48-23411754031827/pip'"]
<198.58.115.201> EXEC ['sshpass', '-d9', 'ssh', '-tt', '-q', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/tmp/ansible-ssh-%h-%p-%r', '-o', 'Port=22', '-o', 'GSSAPIAuthentication=no', '-o', 'PubkeyAuthentication=no', '-o', 'User=root', '-o', 'ConnectTimeout=999999', '198.58.115.201', u'/bin/sh -c \'sudo -k && sudo -H -S -p "[sudo via ansible, key=ttlhctamxfmseklxzjnlbwbbtkptumux] password: " -u myuser /bin/sh -c \'"\'"\'/usr/bin/python /tmp/ansible-1376450420.48-23411754031827/pip\'"\'"\'\'']

pip log:

 it looks like the install hangs when it gets here

Downloading/unpacking xhtml2pdf==0.0.3 (from -r /home/myuser/projects/myapp/deps/production.txt (line 15))

  URLs to search for versions for xhtml2pdf==0.0.3 (from -r /home/myuser/projects/myapp/deps/production.txt (line 15)):
  Could not fetch URL https://pypi.python.org/simple/xhtml2pdf/0.0.3: HTTP Error 404: Not Found (xhtml2pdf/0.0.3)
  Will skip URL https://pypi.python.org/simple/xhtml2pdf/0.0.3 when looking for download links for xhtml2pdf==0.0.3 (from -r /home/myuser/projects/myapp/deps/production.txt (line 15))
  Analyzing links from page https://pypi.python.org/simple/xhtml2pdf/

Michael DeHaan

unread,
Aug 14, 2013, 9:44:18 PM8/14/13
to ansible...@googlegroups.com
Sounds like this is just a problem with the remote source timing out?

Remember to use {{ foo }} vs $foo BTW :)




--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.



--
Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

Jason brooks

unread,
Aug 15, 2013, 12:13:34 AM8/15/13
to ansible...@googlegroups.com
Thank you for the response.   I initially thought it was a timeout issue but i have been able to pip install my requirements file multiple times manually on the remote server.  I also wrote a python script that parsed my requirements file and pip installed each package individually.  I called the script in my playbook and got it to work. But I rather not use the script as a workaround for the pip module.

Michael DeHaan

unread,
Aug 15, 2013, 8:30:21 AM8/15/13
to ansible...@googlegroups.com
There are lots of people using the pip module, so I don't know what's going on on your end.

If you can help debug using hacking/test-module in the checkout that would be useful.

Jason brooks

unread,
Aug 16, 2013, 12:27:41 AM8/16/13
to ansible...@googlegroups.com
i will try that.  I have narrowed it down to these two packages in my requirements file...

paramiko>=1.7.7.1

the pip install does not even attempt to install these packages..as there is no output in the pip log when I try these two individually as the only packages in the requirements file.  However, i was able to get my entire requirements file to install successfully including these two packages when i tested it using localhost.  It only seems to cause an issue on a remote machine and when they are in a requirements file.  I have attempted it on both digital ocean and linode. 

Going to keep looking at my environment and try from a different machine.

I appreciate your help

Matthias Kiefer

unread,
Sep 10, 2013, 8:00:57 AM9/10/13
to ansible...@googlegroups.com
I had a similar problem that pip failed to reach pypi.python.org completely when called from ansible. The reason was that a proxy was needed on the remote host and the proxy to connect to was set in the environment variables http_proxy and https_proxy. Since sudo resets the environment for security reasons, the pip command run from ansible with sudo was not able to connect to pypi.python.org.

You can solve this by adding the following line to the sudoers config:
Defaults        env_keep += "http_proxy https_proxy"

Michael DeHaan

unread,
Sep 10, 2013, 8:35:50 AM9/10/13
to ansible...@googlegroups.com
See the "environment" parameter in Ansible for how to pass an arbitrary environment to any task.


This is better because it doesn't rely on any pre-existing user configuration on the system, or trusting the user's configuration.



--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply all
Reply to author
Forward
0 new messages