Shortcut for making Ansible groups from EC2 tags

[Quentin Stafford-Fraser]

I spent a long time trying to get this to work, and eventually managed it, so just in case it's useful for anyone else...

I wanted to tag my EC2 instances with, say 'webserver: True' and have them then appear in the 'webserver' host group locally, but I wanted to do it as a general case, without having to write special code for every group.  

The ec2.py inventory script will put each host into a group called 'ec2' and also, for example, into 'tag_webserver_True'.  

So this is what I did next:

---

- hosts: ec2

  remote_user: ubuntu

  serial: 1

  tasks:

    - name: Add hosts in EC2 tag groups to local groups

      local_action: add_host

        name={{inventory_hostname}}

        groupname={{item[4:-5]}}

      when: item.startswith('tag_') and item.endswith('_True') and (inventory_hostname in groups[item])

      with_items: group_names

It took me a couple of hours playing with all sorts of variations on this theme to realise that the really important bit is to include 'serial: 1'.  

Adding things to groups using add_host does not work reliably if there are multiple hosts being processed simultaneously.  You will probably find that a single host gets added to the group but no more.

This, however, seems to work fine, and can easily be included in playbooks where it will be ignored if you're not using any EC2 hosts.

Hope that's useful for someone!

[Michael Peters]

Another option I've used is to modify the ec2.py that comes in the
ansible sample inventories. For something simple like this it's not a
big deal, but when you start having more custom groups based on tags
and you don't want the ec2_tag_ prefix on every group (or are mixing
EC2 and non-aws instances), it starts to become more of a pain. It's a
pretty simple python script that's easy to modify. Once we made that
small leap our playbooks became a lot clearer imo.

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/b39f45aa-b6b8-4836-ab65-82b49c48ea95%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Quentin Stafford-Fraser]

I added a note to the add_host docs about this and posted a pull request, but it was rejected on the basis that:

• add_host is non persistent, it only adds hosts to the in memory inventory for the current run, also it bypasses the host loop so it does NOT run on multiple hosts. Due to it's nature it makes little sense to run other than on the master.

which all makes sense, but I think then that a note in the docs to indicate that would be helpful!   How are we to know, otherwise?  More information is always good.

 

It's not just when running on multiple hosts that there's a problem, by the way: it also doesn't work if you use local_action or delegate to localhost when the target for the play includes multiple hosts.

Just my tuppence-worth.

[Quentin Stafford-Fraser]

Ah - and Matt Martz has pointed me at this pull request which says a bit more than mine did.  Good stuff...!

https://github.com/sivel/ansible-modules-core/commit/dcd6ab735abba283cba22d345e233b57ac47f8ac

 

[George Chigrichenko]

Quentin, really useful post.

However I had to add two things in order to skip the errors on hosts that were unreachable. I think this might be helpful in case they are not started and you would like to start them later from the ec2 module.

Tried to suppress the stuff using configuration variables

host_key_checking=True

display_skipped_hosts=True

but that still getting the 

PLAY ***************************************************************************

messages, which is really annoying. 

It is going to be a good idea to include this playbook in the beginning so other scripts can reuse the dynamic group(s).

- hosts: ec2

  serial: 1

  gather_facts: false

  ignore_errors: yes

  

  tasks:

    - name: Add host(s) from EC2 tag group to Controller group

      local_action: add_host

        name={{inventory_hostname}}

        groupname='Put_your_new_dynamic_group_name_here'

      #when: item.startswith('tag_Name_') and item.endswith(your_name_variable) and (inventory_hostname in groups[item]) #optional loose matching

      when: item == 'tag_Name_' + your_name_variable and (inventory_hostname in groups[item])

      with_items: group_names

[Quentin Stafford-Fraser]

Ah - interesting idea.  I tend to have separate Python scripts to stop and start servers, mostly for this reason.  But it would be nice to do everything with Ansible...

[ddrak...@gmail.com]

Thanks for the posts, very helpful. A couple things a year and a half later:

The dynamic inventory script (ec2.py) now returns ec2_tag_Name so for a lot of things you can use the when parameter to say something like "when: ec2_tag_Name == your_tag_name"

However, this doesn't work if you want to use run_once and you get lucky enough to have the first instance in the play hosts (ansible_play_batch[0]) not match this when conditional so the task is ran once but skipped. I came up with two solutions that I detailed here: https://groups.google.com/d/msg/ansible-project/iHRjSvoavww/WDvX_zmAAQAJ

In general though, I wonder if it might just be easier/possible/cleaner(?) to use the ec2_instance_facts module, create in-memory groups using add_host, and then use delegate_to with the groups, running everything on localhost. 

Dave