Getting hostvars from an intersection of groups

[Mehul Ved]

I have 4 machines on rackspace - two sets of machines, for production and beta. Each set has a web server and db server. 

Here's what the metadata for each machine would look like

metadata: {

  group: web,

  set: production

}

metadata: {

  group: db,

  set: production

}

metadata: {

  group: web,

  set: beta

}

metadata: {

  group: db,

  set: beta

}

While deploying to machine set beta and group web, I need to get certain variables from machine set beta and group db. I have been trying various ways to ensure I get the right machine. I got a pointer from somebody on IRC to try intersection filter to get the correct machine from the group.

So, I did this

  debug: var='groups['beta'] | intersection(groups['db'])'

The above gives me a string with machine name

"[u'beta_db1']"

I am unable to use it as shown in variables example

  debug: var='hostvars[{{ group['beta'] | intersection['groups['db']) }}][0]]['db_port']

It returns the result as 

"hostvars[[u'beta_db'][0]]['db_port']"

I intend to use this in my vars/main.yml as

app_db_port: <the variable defined for beta db machine>

Am I in the right direction? If so how to handle the issue where hostvars needs a list but intersection returns a string? Or is there a better way to achieve what I want?

 

--

With Regards,
Mehul Ved

[Brian Coca]

actually [u'beta_db1'] is a list with 1 string item inside, the debug
construct you are using is mixing templated and non templated vars,
that won't work also you seem to have mismatched list indecies.

debug: msg="{{hostvars[group['beta'] |
intersection(groups['db'])[0]]['db_port']}}"
or
debug: var=hostvars[group['beta'] | intersection[groups['db'])[0]]['db_port']

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/CADTL0oYm9Uziv4em%3DYgSxY7SR3MG72yUme%2BJmfy0Q8%3DsXCG2Zg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.



--
Brian Coca

[Mehul Ved]

Hi Bryan,

 Here's what I get when I run the command

$ RAX_CREDS_FILE=.raxpub ansible web -i inventory/rax.py -m debug -a "msg={{ hostvars[groups['beta']|intersect(groups['db'])[0]]['db_port'] }}" --limit beta -v

beta_web1 | FAILED => Failed to template msg={{ hostvars[groups['beta']|intersect(groups['db'])[0]]['db_port'] }}: template error while templating string: expected token ',', got '['

$ RAX_CREDS_FILE=.raxpub ansible web -i inventory/rax.py -m debug -a "msg={{ hostvars[groups['beta']|intersect(groups['db'])[1]]['db_port'] }}" --limit beta -v

beta_web1 | FAILED => Failed to template msg={{ hostvars[groups['beta']|intersect(groups['db'])[1]]['db_port'] }}: template error while templating string: expected token ',', got '['

$ RAX_CREDS_FILE=.raxpub ansible web -i inventory/rax.py -m debug -a "msg={{ hostvars[groups['beta']|intersect(groups['db'])[2]]['db_port'] }}" --limit beta -v

beta_web1 | FAILED => Failed to template msg={{ hostvars[groups['beta']|intersect(groups['db'])[2]]['db_port'] }}: template error while templating string: expected token ',', got '['

notice the increment in the index but it still gets the same error.

If I omit the index to see what is there,

$ RAX_CREDS_FILE=.raxpub ansible web -i inventory/rax.py -m debug -a "msg={{ hostvars[groups['beta']|intersect(groups['db'])]['db_port'] }}" --limit beta -v

beta_web1 | FAILED => One or more undefined variables: ansible.runner.HostVars object has no element [u'beta_db1']

I'm not sure what I'm missing out here.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAJ5XC8m_gsuYNH2hE74v5AVnpO2XquUf6aODzvRpToDWf6mfxg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

With Regards,
Mehul Ved

[Mehul Ved]

I've been trying to look up more on this but it seems like a dead end. In fact, I see a few more question where people are using dynamic inventory and multiple groups facing similar issues. I realized this isn't the only place I'll have an issue, these kind of issues will also show up when working with group_vars.

Is this not a recommended way of doing things? I've found atleast 3 other questions on similar lines.

 - http://www.mattzone.com/ansible/ansible_20150821.html (by tupty)

 - https://groups.google.com/forum/#!topic/Ansible-project/QuqXUV_BEMM

 - https://www.mail-archive.com/ansible...@googlegroups.com/msg08024.html

Is there a better way to handle grouping of servers? Is there anything in the documentation or best practices that I've missed?

[K Cheng]

Hey Mehul,

I'm using dynamic inventory with AWS ec2 module. I had a quick glance at your question so forgive me if I have the wrong idea in what you are asking.

I had the scenario where I needed to get the number of instances that were in the same environment and of the same role type and perform a count of those instances.

I managed to achieve it using this syntax (hopefully this will be of some help to you for your scenario):

- name: Set a count variable of existing instances in the same environment AND role type

  set_fact:

    start_count: "{{ groups['tag_environment_' + env] | default('') | intersect(groups['tag_role_' + instance_role] | default('')) | length}}"

This allows me then tag instances in the same environment and the same role type with an incrementing number: 

- name: Tag instances using the correct sequence number for that environment and role

  ec2_tag:

    profile: "{{ aws_account_name }}"

    region: "{{ region }}"

    resource: "{{ item.id }}"

    tags:

      Name: "{{ instance_role }}{{ '%02d'|format(item.ami_launch_index|int + start_count|int + 1) }}.{{ env }}.aws-{{ region_shortname[region] }}.{{ tld_name_internal }}"

  with_items: "{{ ec2.instances | default({}) }}"

For example, when I launch 2 more instance with environment of stage and role of web, and there were already 2 existing instances, then I would get web03.stage,domain and web04.stage.domain as the tag.

Hope this helps you with your scenario.

Karen

[K Cheng]

Also, further to the utilisation of dynamic inventory, I created a role specifically for running the dynamic inventory as part of a play. I then include this role as a role dependency where needed.

As part of the inventory refresh - I use the output to set a fact that I can then utilise for various other things:

- name: Running AWS inventory refresh

  command: python {{ playbook_dir }}/inventory/{{ aws_account_name }}/ec2.py --refresh-cache

  register: inventory_output

  delegate_to: localhost

- name: Set output of current dynamic inventory

  set_fact: 

    dynamic_inventory: "{{ inventory_output.stdout|from_json }}"

[Mehul Ved]

Hi Karen,

  I already have these machines up and running and correctly tagged. I want to access hostvars of a machine from another machine. e.g.

 - I am on production web machine and need to get the database port, which is defined on db machine.

 - There are 2 db machines, production and beta, both of which have these variables.

In this scenario, how do I ensure that I'm accessing the database port variable from the correct machine?

If I define a variable in a role in my web playbook as

  app_db_port = hostvars[group['database']][0]['db_port']

then I'm not sure if I'm getting the db port of production machine or beta machine.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6f009b56-7f9e-4ba5-9d3b-4468053ee85d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dan Rough]

Mehul / Karen,

Did you ever get anywhere with this? I have just arrived at the same problem and am looking for a solution too.

When I run:

ansible prod -i ./inventories/rax.py -c local -m debug -a var="{{ hostvars[groups['prod']|intersect(groups['monitoring_servers'])][0].rax_addresses['internal-network'][0].addr }}"

I receive the error:

"msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: ansible.vars.hostvars.HostVars object has no element [u'prod-LON-mon-01']"

The intersect has correctly identified the server I wish to single out, prod-LON-mon-01 so I think I'm pretty close to solving this myself. If I hardcode that value in as hostvars['prod-LON-mon-01'] etc, etc, I get the IP address I am after.

I'm sure there's an obvious mistake that I've made, if you, or anyone else for that matter is able to spot it and point it out, I'd be grateful.

Thanks!

Dan.

[Josh Smift]

If you want to solve this a very different way, ansible-quartermaster has
a "metagroups" feature that allows you to create groups of hosts in
inventory based on various criteria, like the value of a variable; so you
can do things like get a prod-web group for the hosts that are in both
prod and web, and more generally a <stack>-<role> group for every host
that has a stack and a role.

See https://github.com/caredotcom/ansible-quartermaster for more.

-Josh (j...@care.com)

(apologies for the automatic corporate disclaimer that follows)
This email is intended for the person(s) to whom it is addressed and may contain information that is PRIVILEGED or CONFIDENTIAL. Any unauthorized use, distribution, copying, or disclosure by any person other than the addressee(s) is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and delete the message and any attachments from your system.

[Dan Rough]

Hi,

Immediately after sending this mail I got an answer to this question from another source - a user called sivel on the #ansible IRC channel. I thought that I would put the answer here in case others hit a similar problem.

I had one of the [0] in the wrong place, which meant that I was looking for a key of [u'prod-LON-mon-01'] instead of u'prod-LON-mon-01'. The solution that Sivel gave me was to rewrite what I had as:

{{ hostvars[groups['prod']|intersect(groups['monitoring_servers'])|first].rax_addresses['internal-network'][0].addr }}

He also included the use of |first which makes the selection clearer to read which is a bonus too.

Hope this helps someone else.

[Dan Rough]

Thanks Josh. That's useful to know about. I have one of the situations that the README describes - I have some kit at Rackspace, and some on a dedicated server elsewhere. At the moment, I store all the inventory files in a directory, inventories. When I run a playbook I provide the directory as the location of the inventory, rather than the specific inventory file. for e.g. ansible-playbook deploy.yml -i ./inventories/. Can you help me understand how AQ assists further on top of that ability? It's not immediately apparent to me and I don't want to miss something really useful!