Dynamically creating a list in a group_vars yml file

[Kristopher Cieplak]

I am wondering if this is possible I have a group_vars file with some variables in it, and I need to set, the list is generated from the inventory and a mapping to the ec2 private ip address.  

For example in my group_vars file:

cassandra: {keyspace: default, log_time: false, hosts: [ "{% for host in groups['tag_role_cassandra_true'] %}{{hostvars[host].ec2_private_ip_address}}{% if not loop.last %},{% endif %}{% endfor %}" ]}

This ends up being the following if I dump it using the "debug" directive:

"msg": "cassandra {'keyspace': 'default', 'log_time': False, 'hosts': [u'172.31.2.85,172.31.2.84,172.31.2.86,172.31.2.83']}"

As you can see that is a list of one string. I want a list of all hosts, in this case four hosts, i.e it should be:

'hosts': ['172.31.2.85', '172.31.2.84' ,'172.31.2.86', '172.31.2.83']

Does anyone know anyway to loop over all the hosts do the conversion to private ip and append to the "list".

My fallback is to generate it as a string, then use that lower down , but would be nice to have the list, as I can output it in various templates differently.

Thanks

[Michael DeHaan]

This seems to be a very un-ansible-like bad idea and you shouldn't do it.   

In general, Ansible is not meant to be a programming language, and inventory is meant to be something edited by humans or pulled from a source of authority -- I'm probably not understanding why you want to do this though.

Let's step back here -- what's the use case for needing to do something like this?

Give us more info about the real world thing you are trying to model, and we can likely share some more Ansible-like suggestions.

--Michael

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/572e3dba-2fc4-4f6b-8a7a-de506a1f02c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kristopher Cieplak]

Here is fairly generic example:

We have N zookeepers in our datacenter, if we want more redundancy, we add a new instance with a tag indicating it is has a "zookeeper" role.  Now I can re-run my playbook which will build configurations to list all those zookeepers where needed.  Lets assume one configuration file requires the format to be "1.2.3.4, 4.5.6.7" yet another requires "1.2.3.4:8080,4.5.6.7:8080". Also depending on my environment these address may be private or public. I want to per environment set the list of hosts and then in the lower level playbooks format it properly. 

If I put all this logic in the lower level playbooks/tempates that generate the configs, I will need to put a condition to use the private or public addresses also tying the playbook to ec2 as an example.  All my lower level playbooks are generic, give them a list of hosts and that is it, complexity is at the top.  If I switch to another cloud provider I can modify the top level variables and I am done.

Anyways I figured out how to do this by using "do", these are in my "group_vars" for my production and staging environments

Production AWS environment

zookeeper:

  hosts: "{% set hosts = [] %}{% for host in groups['zookeeper_master'] %}{% do hosts.append(hostvars[host].ec2_private_ip_address) %}{% endfor %}{{hosts}}"

Staging Environment 

zookeeper:

  hosts: "{% set hosts = [] %}{% for host in groups['zookeeper_master'] %}{host}{% endfor %}"

I am very interested/scared of your comment that ansible is not meant to be a programming language, and that is un-ansbile-like.  The world we have today is very dynamic, clusters grow and shrink and need to be reconfigured any time that happens.  I personally like the simplicity of ansbile compared to things like chef or salt, but there are days when trying to debug jinja, makes me want to jump out a window.  I personally have 20 years of programming behind me so doing things programmatically is a concept that I cannot shake easily.

Thanks for your input

Kristopher

[Michael DeHaan]

That loop and if statement in the variable file is also very very un-Ansible like:

Re this part:

"We have N zookeepers in our datacenter, if we want more redundancy, we add a new instance with a tag indicating it is has a "zookeeper" role.  Now I can re-run my playbook which will build configurations to list all those zookeepers where needed.  Lets assume one configuration file requires the format to be "1.2.3.4, 4.5.6.7" yet another requires "1.2.3.4:8080,4.5.6.7:8080". Also depending on my environment these address may be private or public. I want to per environment set the list of hosts and then in the lower level playbooks format it properly. "

- hosts: zookeepers

  tasks:

     - shell: do something with {{ item }}

       with_items: groups.webservers

This is a way to run something on each zookeeper with the name of each webserver, you could also invert this pattern to do something on each webserver with a reference to each zookeeper.

Though ultimately I'll still say I don't completely understand the question, but if you have a per-environment list of hosts, like stage vs production, keeping different variables and lists of hosts for them, and different inventories, is always a good practice.

If you are using Jinja2 too much, it usually means you are over-using Jinja2, and should instead use the primatives that Ansible provides.

If conditionals in Ansible playbooks that use Jinja2 are not something in intended usage, simple variable substitution is what they are there for.

Templates can obviously get more complicated, but it's usually users coming from Chef that try to program in YAML and have a bad experience.

Ansible wishes to describe processes, but it's not programming.

Apologies if I'm talking past you here, but Ansible does have a certain aesthethic and it feels you are trying to hammer something in sideways a little bit.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/8fc3b951-3332-4360-bd0b-767b93a80df8%40googlegroups.com.

[Lee Connell]

\\Production AWS environment

zookeeper:

  hosts: "{% set hosts = [] %}{% for host in groups['zookeeper_master'] %}{% do hosts.append(hostvars[host].ec2_private_ip_address) %}{% endfor %}{{hosts}}" \\

What would be the Ansible way to generate a list as what is being done here? You provided an example of looping through shell commands, but what is required is assigning a list variable to the ip addresses of each host.