Can't find boto for ansible EC2 module

[Dan McKean]

I've been considering Ansible for creating our EC2 instances, and have run into a problem with ansible not finding boto from either the command line or a playbook.

Here's the command line and response; the playbook is similar:

ansible localhost -m ec2 -a "image=ami-0358ce33 instance_type=t1.micro keypair=myKeypairName group=default wait=true"

localhost | FAILED >> {

    "failed": true,

    "msg": "boto required for this module"

}

I'm running Ansible from an activated virtualenv.  Ansible was installed with pip install ansible, and boto's installed as well in the venv (as well as globally).

If I activate the venv and open a command prompt, I can import boto without any problem, and a boto.connect_ec2() succeeds (there is a ~/.boto config file present).  Importing ansible and creating a Runner works succeeds as well.  I'm running this on Mac OS X.

Any suggestions on how to fix this?   Thanks!

-- Dan

[Michael DeHaan]

Ansible is not going to find boto in your virtualenv, so you need to install boto into site packages properly.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[James Martin]

In your hosts file, set your ansible_python_interpreter= the path of
your python intepreter in your virtual env, it will work. For
example:

[localhost]
127.0.0.1 ansible_python_interpreter=/Users/jmartin/.virtualenvs/ansible-git/bin/python

On Fri, Aug 2, 2013 at 10:52 AM, Dan McKean <danlm...@gmail.com> wrote:

[Michael DeHaan]

That too, but less to remember if you just install it like an app instead of realizing it's a Python application and treating it special.

I use LOTS of apps without knowing what language they are written in.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Dan McKean]

Thanks, Michael!

boto is also install globally with pip and working fine.

What is the proper way to install boto for use by ansible on Mac OS X?

Dan

[Dan McKean]

Thanks for your response.

As a practical matter for the environments and admin tools I develop, I can't install Python applications and packages globally; they must be run in a virtual environment.  That's very common usage (our Django and Flask apps are virtualenv aware), and I'm a bit disappointed that I can't just get Ansible up and running in a virtual environment... it'd be great if that were a bit more obvious in the docs, or there's a blog post to point this simple usage out.

That being said, Ansible is a breath of fresh air...  I've already been able to do quite a bit in just a few hours that would've taken me days with Chef and Puppet.

Thanks,

Dan

[Michael DeHaan]

Good to hear!

Yeah see James's reply about ansible_python_interpreter.

It solves the problem too.

[Michael DeHaan]

I generally would do:

sudo pip install boto

[Peter Sankauskas]

On OSX, there are many ways to use Python, and as you try to overwrite the default 2.6 that comes with OSX, things can get messy.

The best way I have found is using brew and pip. You will need to change your PATH so that brew's install location is before the others, but essentially, it is:

$ brew install python

$ pip install boto --upgrade

Peter Sankauskas

[Chris Shenton]

On Friday, August 2, 2013 11:16:08 AM UTC-4, James Martin wrote:

In your hosts file, set your ansible_python_interpreter=  the path of
your python intepreter  in your virtual env, it will work.  For
example:

[localhost]
127.0.0.1 ansible_python_interpreter=/Users/jmartin/.virtualenvs/ansible-git/bin/python

Perfect, thanks for this. 

[Charles Mims]

FWIW, I have boto installed globally using pip:

sudo pip install boto

I can import boto into python without issue, but I get this same error message.  I'm using OS X mavericks.

[cos...@gmail.com]

Hi,

I came across the same issue when I try to run cloudformation module in CentOS in python virtual env. (it used to work some time ago):
(venv)[root@test ansible]# ansible-playbook aws-cf-provision.yml -vv
...
failed: [localhost] => {"failed": true, "parsed": false}
failed=True msg='boto required for this module'
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1429011194.43-47297413805384/cloudformation", line 130, in <module>
    sys.exit(1)
NameError: name 'sys' is not defined

Setting up "export PYTHONPATH=/root/venv/lib/python2.6/site-packages" or "localhost ansible_python_interpreter=/root/venv/bin/python" in hosts makes the original error disappear but the resource doesn't get created ...

Here is the current error:

(venv)[root@test ansible]# ansible-playbook aws-cf-provision.yml -vv

PLAY [localhost] **************************************************************

TASK: [Create VPC, internet gateway, route tables and security groups] ********
<localhost> REMOTE_MODULE cloudformation template=VPCStack.json state=present region=eu-west-1 stack_name=VPCStack-dev2 aws_access_key=******************************* aws_secret_key==*******************************
failed: [localhost] => {"failed": true}

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/root/aws-cf-provision.retry

localhost                  : ok=0    changed=0    unreachable=0    failed=1


(venv)[root@test ansible]# pip list
ansible (1.9.0.1)
argparse (1.3.0)
awscli (1.7.18)
bcdoc (0.13.0)
boto (2.38.0)
botocore (0.101.0)
colorama (0.3.3)
docutils (0.12)
ecdsa (0.13)
httplib2 (0.9)
Jinja2 (2.7.3)
jmespath (0.6.1)
List (1.3.0)
MarkupSafe (0.23)
ordereddict (1.1)
paramiko (1.15.2)
pip (1.4.1)
pyasn1 (0.1.7)
pycrypto (2.6.1)
python-dateutil (2.4.2)
PyYAML (3.11)
rsa (3.1.4)
setuptools (0.9.8)
simplejson (3.6.5)
six (1.9.0)


It doesn't even point out where the problem could be. Is there any way to find out?

PS: I tried to delete the python virtualenv and recreate it back but the issue still remains.

Regards,
Constantin

[Justin Phelps]

I managed to fix this with these two tasks:

  - name: Find the path to the python interpreter

    command: which python

    register: pythonpath

    when: lookup('env', 'CI') == "true"

  - name: Set python interpreter path when in CI

    set_fact: ansible_python_interpreter={{ pythonpath.stdout }}

    when: lookup('env', 'CI') == "true"

You would take out the when statements as they are specific to my testing environment.

[James Martin]

Yay, 2 year old thread :).

The way you solve this is by modifying your inventory for localhost with the proper setting:

#inventory file
localhost ansible_python_interpreter="/usr/bin/env python"

This will automatically pick up the active virtualenv's python interpreter and modules that you've installed there, including boto.  You don't need to run any tasks to discover the path to the python interpreter.

[Justin Phelps]

Thanks James.

That is easier and I can confirm it works in CircleCI.

[Alex Leonhardt]

I tried a pull request for the GCE modules (as example) to use /usr/bin/env python instead of a fixed /usr/bin/python but that was dismissed without real consideration of whether it actually be a good idea (or not) nor a detailed explanation as to why it was considered "bad".

I don't see why, when /usr/bin/env python will mostly be the correct python, it's still being enforced to use /usr/bin/python :/ .

Maybe someone here knows the inner details as to why??

Alex

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/859b0572-7b3a-4716-99d1-6ef5134eb10a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Timothy Appnel]

Ansible doesn't use a fixed /usr/bin/python necessarily. If you set a different python interpreter via your inventory as Justin did here and keep the remotes files, you'll notice that Ansible has replaced with /usr/bin/python with the python interpreter you have in your inventory.

> I don't see why, when /usr/bin/env python will mostly be the correct python, it's still being enforced to use /usr/bin/python :/ .

The keyword in your sentence here is "mostly." Ansible needs to be using the intended python interpreter "always." I've worked in environments where the default python was way out of date and the modern version was in an alternate location. Because Ansible was forced to use a service account that did not have a profile to alter the PATH, we were forced explictly specify the python interpreter and let Ansible do it's thing. I've also worked in environments where python was in different alternate locations.

Think of #!/usr/bin/python as just a placeholder, albeit it a functional one if left as is.

Hope that helps answer your question. 

<tim/> 

Timothy Appnel Senior Solutions Architect Ansible, Inc. • www.ansible.com

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CACSH3MvAr3t-jbQmBYAw4fMwt1w9S-6pZNcCQdRQ8oOi8M0xJg%40mail.gmail.com.

[Troy Cauble]

What can I do if I'm not using an inventory file?  I'm exec-ing from a script with "ansible-playbook -i <host>, ...".

Also, the ansible and ansible-playbook scripts in my_virtualenv_dir/bin start with

#!/my_virtualenv_dir/bin/python2

So they're using the right python to start...

[Troy Cauble]

I found the solution elsewhere you can set this via -e

   -e 'ansible_python_interpreter="/usr/bin/env python"'