Traceback on group_by

[Mark Casey]

Hello,

I'm getting a traceback on a group_by, which is intended to create a group named after each of a host's existing groups, prepended by a variable that is being set with '-e' at invocation (a site/host-group ID):

- name: vagrant | Create provider-agnostic ID_and_type group
  group_by: key="{{ id }}_{{ item }}"
  with_items: group_names
  when: provider == "vagrant"

The variable provider is also being set via '-e'. So for example, a host that was part of the dns and mail groups would be added to groups like prodSite1_dns and prodSite1_mail. These are not being set statically in an inventory file because sometimes it is run on ec2 hosts and the groups are pulled from tags, and sometimes it is run on a vagrant box and the groups are written into an inventory script.

Here is what I'm getting when running with -vvvv:

TASK: [iaas_gather_multi | ec2 | Create provider-agnostic ID group] ***********
skipping: [localhost]

TASK: [iaas_gather_multi | ec2 | Create provider-agnostic type group] *********
skipping: [localhost]

TASK: [iaas_gather_multi | ec2 | Create provider-agnostic ID_and_type group] ***
skipping: [localhost]

TASK: [iaas_gather_multi | ec2 | Create provider-agnostic host-provisioned boolean] ***
skipping: [localhost]

TASK: [iaas_gather_multi | debug var=group_names] *****************************
ok: [localhost] => {
    "group_names": [
        "db",
        "webnode"
    ],
    "item": ""
}

 TASK: [iaas_gather_multi | vagrant | Create provider-agnostic ID_and_type group] ***
created 'group_by' ActionModule: key={{id}}_{{item}}
changed: [localhost] => (item=db) => {"changed": true, "groups": {"vagrant-devhost_db": ["localhost"]}, "item": "db"}
created 'group_by' ActionModule: key={{id}}_{{item}}
fatal: [localhost] => Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/ansible/runner/__init__.py", line 394, in _executor
    exec_rc = self._executor_internal(host, new_stdin)
  File "/usr/lib/pymodules/python2.7/ansible/runner/__init__.py", line 514, in _executor_internal
    complex_args=complex_args
  File "/usr/lib/pymodules/python2.7/ansible/runner/__init__.py", line 685, in _executor_internal_inner
    result = handler.run(conn, tmp, module_name, module_args, inject, complex_args)
  File "/usr/lib/pymodules/python2.7/ansible/runner/action_plugins/group_by.py", line 86, in run
    del self.runner.inventory._vars_per_host[host]
KeyError: u'localhost'


FATAL: all hosts have already failed -- aborting

Thank you,
Mark

[Mark Casey]

For what it's worth, I tried taking out the reference to group_names and instead passed into the role as a variable a list of group names to test with. I still got the traceback. It doesn't happen if I just duplicate the task and use multiple copies of it that are even more hard coded such as: "{{ id }}_aStaticGroupName". Now the whole thing breaks horribly past that point because of the change, but the tracebacks stop.

So AFAICT I get the traceback trying to use "{{ id }}_{{ item }}".

Also I forgot to mention that I'm seeing this on v1.4.3.

Thank you,
Mark

[James Tanner]

Please file a bug for any tracebacks you find. We would definitely like to know if you can reproduce this on 1.4.4 or on the latest devel.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Matt Coddington]

Hi Mark/James,

Do you guys know if there was a bug filed on this?  I'm running into a similar thing in 1.5.3.  If not i'll try and put together a minimal playbook to reproduce.

thanks,

matt

[Mark Casey]

At the time I wasn't immediately able to test the newer version James mentioned, and I worked around it soon after. So in short, no, I forgot to file anything in this case (sorry!).

How similar is your issue? (mainly curious if it is a host grouping thing, or if you're just also iterating a built in structure with with_items:)

My normal "real" ec2 workflow expected Ansible to use a dynamic inventory anyway, and I'd already parameter-ized that call to allow for different inventory scripts in case I wanted to add rax support later. So what I ended up doing to get by was get help on IRC to write a simple (basically fake) dynamic inventory script for vagrant deploys that just prints out "localhost", but already a member of the groups I needed. So it's pretty much just a static inventory file in a format that is more convenient to the situation.

# vagrant.py

#!/usr/bin/env python

print '{"db": {"hosts": ["localhost"], "vars": {"ansible_connection": "local"}}, "vagrant_dummy_group": {"hosts": ["localhost"], "vars": {"ansible_connection": "local"}}, "web": {"hosts": ["localhost"], "vars": {"ansible_connection": "local"}}}'

Thanks,
Mark

[Matt Coddington]

Hey Mark,

Mine is using with_items as well; i think that's the common thread here.  Specifically the task that seems to cause the exact traceback you see is as follows:

- name: generate role groups

  group_by:  key="{{ item }}"

  with_items: all_roles

and it throws this traceback:

fatal: [waluat1] => Traceback (most recent call last):

  File "/rbi/ansible/ansible/lib/ansible/runner/__init__.py", line 532, in _executor

    exec_rc = self._executor_internal(host, new_stdin)

  File "/rbi/ansible/ansible/lib/ansible/runner/__init__.py", line 654, in _executor_internal

    complex_args=complex_args

  File "/rbi/ansible/ansible/lib/ansible/runner/__init__.py", line 809, in _executor_internal_inner

    result = handler.run(conn, tmp, module_name, module_args, inject, complex_args)

  File "/rbi/ansible/ansible/lib/ansible/runner/action_plugins/group_by.py", line 86, in run

    del self.runner.inventory._vars_per_host[host]

KeyError: 'waluat1'

I will file something on github with a small example to reproduce.

matt

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6ea7d50b-64d9-426d-8542-041b07ddaed4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Coddington]

I've opened https://github.com/ansible/ansible/issues/6660 with details... 

thanks!
matt

[Matt Coddington]

Thanks James for the patch!  I can verify that https://github.com/ansible/ansible/commit/dc658eaa1c636c47804bf70eddc55ada5e4078bf, when applied manually to 1.5.3 fixes my issues here!

matt