Add hosts to the inventory by their Name (or other) tag in ec2

[Hagai Kariti]

Related to PRs: https://github.com/ansible/ansible/pull/6721 and https://github.com/ansible/ansible/pull/3697

I know you rejected similar changes in the past, but I think seeing some indicative hostname in playbook runs is very useful. I generate my machine's hostname on creation (so they're still 'cattle'), and 'prod-nginx-3' is more useful than "ec2-xx-xx-xx.kill.me.now.com". :)

I modified the ec2 inventory script and added a 'hostname_variable' config option, that changes the name the host is added to the inventory with. It can use tags or instance variables. The addressing still uses destination_variable, and the default behavior if not defined is to use the destination_variable.

Just wanted to ask if the previous rejections are due to bad implementation or conscious decision against the idea. Also ideas for improvement are welcome.

Thanks.

[Michael DeHaan]

It's fine if you want to modify ec2.py to do something different for your environment -- doesn't mean it's right for core, however.

This part:

" The addressing still uses destination_variable, and the default behavior if not defined is to use the destination_variable."

I have a hard time parsing this.

--
You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hagai Kariti]

Hi Michael, thanks for the reply.

On Tuesday, May 6, 2014 6:45:00 AM UTC+3, michael wrote:

It's fine if you want to modify ec2.py to do something different for your environment -- doesn't mean it's right for core, however.

 

Well, of course. But I would prefer to see it in core as I think I'm the only once who needs it, hence my questions about the reasons similar changes were rejected in the past. :) 

This part:

" The addressing still uses destination_variable, and the default behavior if not defined is to use the destination_variable."

I have a hard time parsing this.

Gaaah, forgot to end that sentence. The default is to use destination_variable for the name as well. Example:

destination_variable = public_dns_name

hostname_variable = tag_Name

inventory_name will be according to the Name tag, while ansible_ssh_host will be the instance's public DNS name. If the tag isn't defined or the whole config line is commented out, the public DNS name will be the inventory_name as well.

[Hagai Kariti]

On Tuesday, May 6, 2014 10:18:31 AM UTC+3, Hagai Kariti wrote:

Hi Michael, thanks for the reply.

On Tuesday, May 6, 2014 6:45:00 AM UTC+3, michael wrote:

It's fine if you want to modify ec2.py to do something different for your environment -- doesn't mean it's right for core, however.

 

Well, of course. But I would prefer to see it in core as I think I'm the only once who needs it, hence my questions about the reasons similar changes were rejected in the past. :) 

*not the only once.. proofreading after waking up not a good idea.

[Michael DeHaan]

Your explanation helps me understand a good bit.

As long as the documentation explains what this does and the defaults do what things do now.

However, I'm still not entirely sold on how many people would use a tag to assign the hostname, to make it generally applicable, so a bit on the fence yet.

[Will Thames]

We also use the Name tag for hostnames. As long as hostname_variable could be tag_Whatever (i.e. not restricted to just Name) this could be very useful to us too.

Will

[Will Thames]

We also use the Name tag for hostnames. As long as hostname_variable could be tag_Whatever (i.e. not restricted to just Name) this could be very useful to us too.

Will

On Wednesday, May 14, 2014 9:55:47 AM UTC+10, michael wrote:

[Peter Gehres]

We have definitely had instances where we wanted to set hostvars in ec2.  What about using tag_hostname_[hostname]?

Similarly, it would be really nice to be able to assign groups with tag_(group|role)_[name] so that we can use the same group vars between cloud providers and bare metal.

Peter Gehres

Site Reliability Engineer | AppDynamics, Inc.

www.appdynamics.com | AS62897

[Hagai Kariti]

A lot of what you want can be done with a combination of static (even empty) inventory for defining group/host vars and the ec2 script. Put them both in a directory and use that as your inventory.

As for assigning groups, the ec2 script groups by ec2 tags already. You can use the static inventory to put the generated ec2 groups as child groups of your bare-metal ones.

[Peter Gehres]

On Wed, May 14, 2014 at 11:33 PM, Hagai Kariti <hka...@gmail.com> wrote:

A lot of what you want can be done with a combination of static (even empty) inventory for defining group/host vars and the ec2 script. Put them both in a directory and use that as your inventory.

If I am understanding this correctly, I've never heard of this. Is there documentation anywhere for this method of inventory?

That seems sub-optimal and not very easy for new users as well as for troubleshooting.  We actually share our roles among many teams in the company where each team has a different ansible-playbooks and AWS account (ops, qa, perf, development, support, etc).  Not being able to have this be automagical when another user utilizes a shared role makes things more difficult. 

This certainly isn't a show-stopper but it would make things a lot more friendly.

As for assigning groups, the ec2 script groups by ec2 tags already. You can use the static inventory to put the generated ec2 groups as child groups of your bare-metal ones.

Oh, we do assign groups with tags.  I am just tired of seeing "tag_"  in my group names and not having these map 1:1 to role names like everything else :-)

Apparently I also forgot that you can't have multiple tags with the same key so that idea also gets more fuzzy (I guess you'd have to do something like tag_group_g1_g2_g3 and then split the value on _).

[Hagai Kariti]

On Thursday, May 15, 2014 10:03:03 AM UTC+3, Peter Gehres wrote:

On Wed, May 14, 2014 at 11:33 PM, Hagai Kariti <hka...@gmail.com> wrote:

A lot of what you want can be done with a combination of static (even empty) inventory for defining group/host vars and the ec2 script. Put them both in a directory and use that as your inventory.

If I am understanding this correctly, I've never heard of this. Is there documentation anywhere for this method of inventory?

It's briefly mentioned here: http://docs.ansible.com/intro_dynamic_inventory.html, at the bottom.

 

That seems sub-optimal and not very easy for new users as well as for troubleshooting.  We actually share our roles among many teams in the company where each team has a different ansible-playbooks and AWS account (ops, qa, perf, development, support, etc).  Not being able to have this be automagical when another user utilizes a shared role makes things more difficult. 

Seems very simple to me:

inventory/

   ec2.py

   ec2.ini

   static

   host_vars/

   group_vars/

is pretty elegant imho. Put group and hostvars in their relevant directories (you can do that now because you have a static inventory file), and define group membership/hierarchy and bare-metal machines in static.

But this is turning into an ansible-project discussion, so let's stop here :)

[Michael DeHaan]

On Thu, May 15, 2014 at 3:03 AM, Peter Gehres <peter....@appdynamics.com> wrote:

On Wed, May 14, 2014 at 11:33 PM, Hagai Kariti <hka...@gmail.com> wrote:

A lot of what you want can be done with a combination of static (even empty) inventory for defining group/host vars and the ec2 script. Put them both in a directory and use that as your inventory.

If I am understanding this correctly, I've never heard of this. Is there documentation anywhere for this method of inventory?

That seems sub-optimal and not very easy for new users as well as for troubleshooting.  We actually share our roles among many teams in the company where each team has a different ansible-playbooks and AWS account (ops, qa, perf, development, support, etc).  Not being able to have this be automagical when another user utilizes a shared role makes things more difficult. 

You can have an inventory directory full of dynamic and file sources, and it acts as one inventory source if you -i the directory.

It's quite useful if you have EC2 and Rackspace in one source, or a mixture of EC2 and on-premise.

If you don't like it, just don't use it.   Don't have to claim it's not easy or anything not even having tried it.

 

As for assigning groups, the ec2 script groups by ec2 tags already. You can use the static inventory to put the generated ec2 groups as child groups of your bare-metal ones.

Oh, we do assign groups with tags.  I am just tired of seeing "tag_"  in my group names and not having these map 1:1 to role names like everything else :-)

I'd suggest you write your own inventory script!  The provided ones are merely an example.

 

[Michael DeHaan]

Relevant pull request to this thread:  https://github.com/ansible/ansible/pull/7395

[Jean-Baptiste Barth]

Hi,

I just found this thread and I can only speak for my own use case, but having the instance name and region in ansible failing runs is quite useful to me, so we can debug later in the AWS console or elsewhere.

I was wondering if you think that "promoting" route53 aliases as the main name of an instance would be a better practice than tags ? I implemented it here: https://github.com/jbbarth/ansible/commit/f66ea2c0be2d9d007c07c04252d777193fcdddee and I'd be very happy to improve it in the next few days/weeks by adding tests or improving the code if you think a merge upstream is possible.

I wanted to have your opinion before submitting a pull request. Let me know what you think.

Anyway thanks for a fantastic tool, keep up the good work!

Cheers,

-- 

Jean-Baptiste

[Chris Church]

That change seems reasonable to me.

Along with the existing PRs for setting an alternate hostname (https://github.com/ansible/ansible/pull/7395, https://github.com/ansible/ansible/pull/7601 and https://github.com/ansible/ansible/pull/9825), it provides another option for certain use cases.

[Jean-Baptiste Barth]

Thanks for the feedback. I opened a PR here: https://github.com/ansible/ansible/pull/9900 ; any other feedback welcome !

Cheers,