ansible 2.1 setup fails with this module requires key=value arguments (['<<INCLUDE_ANSIBLE_MODULE_ARGS>>'])

[Greg Fausak]

I am migrating to 2.1 from 1.9.  I ran into a few issues, but it has been fairly painless. I've run across this, and I have narrowed it down to exactly how to recreate it.  Once I get in this state ansible is not usable anymore.

I've got a playbook, target is ubuntu 15.10, source is mac osx.  i am using vagrant to setup a vm.  then i ssh in modifying the ansible.cfg:

[default]

nocows = yes

retry_files_enabled = False

remote_user = vagrant

deprecation_warnings = False

host_key_checking = False

force_handlers = True

#allow_world_readable_tmpfiles = True

hostfile = /Users/c0faulo/git/vagrant/hosts

[ssh_connection]

ssh_args = -F /Users/c0faulo/git/vagrant/machines/blank/.vagrant_ssh_config -o ControlMaster=auto -o ControlPersist=30m

scp_if_ssh = True

control_path = ~/.ssh/ansible-%%r@%%h:%%p

If it matters, here is my .vagrant_ssh_config file:

Host default

  HostName 127.0.0.1

  User vagrant

  Port 2222

  UserKnownHostsFile /dev/null

  StrictHostKeyChecking no

  PasswordAuthentication no

  IdentityFile "/Users/c0faulo/git/vagrant/machines/blank/.vagrant/machines/default/virtualbox/private_key"

  IdentitiesOnly yes

  LogLevel FATAL

Everything works in the playbook.  When I try to run the playbook again I get this error.  By binary search I found the line that messes up my ansible connection:

I have several repos from my accounts that I install, then I run a command (usually python setup.py install) in that repo.  Here is the task:

    - name: install repos

      become: yes

      become_user: "{{item.user}}"

      shell: "( cd ~{{login}}/git/{{item.repo}}; {{item.install}})"

      with_items:

        "{{ repos }}"

and my repos dictionary:

  vars:

    repos:

      - repo: "gen_yaml"

        org: "VCP"

        install: "echo nothing to do"

        #install: "python setup.py install"

        user: "root"

        url: "http://txslnno-github.cds.eng.vzwcorp.com/VCP/gen_yaml.git"

      - repo: "dhcp-pb"

        org: "VCP"

        install: "echo nothing to do"

        user: "{{login}}"

        url: "http://txslnno-github.cds.eng.vzwcorp.com/VCP/dhcp-pb.git"

      - repo: "cmdb"

        org: "corona"

        install: "make site_torrance"

        user: "{{login}}"

        url: "http://txslnno-github.cds.eng.vzwcorp.com/corona/cmdb.git"

you can see the install: "echo nothing to to"

When I change that to the python setup.py install, everything works, but then I can't run ansible again.

Something appears to get messed up in the ansible environment on the target side.

My program 'gen_yaml' is a fairly simple program.  It needs pyyaml and psycopg2.  I did a manual install and this is the verbose output :

creating dist

creating 'dist/gen_yaml-0.1.9-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it

removing 'build/bdist.linux-x86_64/egg' (and everything under it)

Processing gen_yaml-0.1.9-py2.7.egg

creating /usr/local/lib/python2.7/dist-packages/gen_yaml-0.1.9-py2.7.egg

Extracting gen_yaml-0.1.9-py2.7.egg to /usr/local/lib/python2.7/dist-packages

Adding gen-yaml 0.1.9 to easy-install.pth file

Installing gen_yaml script to /usr/local/bin

Installed /usr/local/lib/python2.7/dist-packages/gen_yaml-0.1.9-py2.7.egg

Processing dependencies for gen-yaml==0.1.9

Searching for psycopg2==2.6

Best match: psycopg2 2.6

Adding psycopg2 2.6 to easy-install.pth file

Using /usr/lib/python2.7/dist-packages

Finished processing dependencies for gen-yaml==0.1.9

My program gen_yaml works, and the installation worked, but I can't access with the ansible anymore?

-g

[Greg Fausak]

In the last 24 hours I looked all over for this.

If I install *anything* with python setup.py install I get a failure.

The hint was the output in the setup.py process:

Adding psycopg2 2.6 to easy-install.pth file /usr/local/lib/python2.7/dist-packages/easy-install.pth

I realized I didn't know what that meant, so I started looking around a little.  It turns out there is a file called easy-install.pth that is read by python at a very low level.  The contents of my file after Ansible stopped working (after my setup.py):

import sys; sys.__plen = len(sys.path)

./cmdb_utils-0.0.4-py2.7.egg

./argparse_config-0.5.1-py2.7.egg

/usr/lib/python2.7/dist-packages

import sys; new = sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p = getattr(sys, '__egginsert', 0); sys.path[p:p] = new; sys.__egginsert = p + len(new)

And the contents of the file before I do a setup.py:

If I edit the file and remove this line:

/usr/lib/python2.7/dist-packages

Then ansible works again.

It has something to do with the order of my apt-gets and my python setup.py installs.  II haven't been able to figure out exactly how to recreate it.

-g

[Roman Gaufman]

Hi Greg,

I'm experiencing the same thing on 2 servers, rest are fine. I can ssh to them and I cannot see anything obviously wrong:

TASK [setup] *******************************************************************

fatal: [0ca87760df7de0fe2d36]: FAILED! => {"changed": false, "failed": true, "msg": "this module requires key=value arguments (['<<INCLUDE_ANSIBLE_MODULE_ARGS>>'])"}

fatal: [33ac8c46007794dee1a6]: FAILED! => {"changed": false, "failed": true, "msg": "this module requires key=value arguments (['<<INCLUDE_ANSIBLE_MODULE_ARGS>>'])"}

to retry, use: --limit @timeboxes.retry

I resolved the problem by moving this file out of the way:

sudo mv /usr/local/lib/python2.7/dist-packages/easy-install.pth /usr/local/lib/python2.7/dist-packages/easy-install.pth.backup

Very strange.

[Adam Grant]

Hi Roman,
I believe you are seeing this:

FAILED! => {"changed": false, "failed": true, "msg": "this module requires key=value arguments (['<<INCLUDE_ANSIBLE_MODULE_ARGS>>'])"}

because those 2 servers are sourcing local out-of-date ansible.egg files (in their easy-install.pth)? It was the case for me.

This seems to be creating a namespace collision for Ansible 2.1 lib imports when the remote python interpreter is initialized to run python code remotely.

-Adam