Append to lists

[Martin Devlin]

Why can't I append to a list so that items in different group_vars, for example, can stack up in the same list depending on which inventory groups a node is a member of.

E.g:

group_vars/chocolate includes:

sauces:

-name : chocolate
value: yummy

group_vars/strawberry includes:

sauces:

-name : strawberry
value: more_yummy

So that in some task I can use sauces to iterate over both?

Using ansible 1.6.0

[Michael DeHaan]

hash_behavior merge would merge hashes, though in your case, what you have is a list of hashes.

Thus hash_behavior doesn't apply.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/8e2f97b3-b127-483d-bc70-ed9421299cb7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Martin Devlin]

Thanks, so is there no way to do this at the moment?

Why I want to this:

I have several kinds of web site definitions and in production they live on separate nodes. However in dev/test I want the same node to host them on and I want to use the combined list in a with_nested. But I can quite easily think of many other applications for this. I just want to be sure it's not supported at the moment and if there's a reason I shouldn't be trying to do it like this.

[Martin Devlin]

[Maciej Delmanowski]

Here's my solution with nginx (role: https://github.com/ginas/ginas/tree/master/playbooks/roles/nginx).

My websites are defined as hashes in inventory variables, like this:

inventory_nginx_site1:

  default: False

  name: 'subdomain.{{ ansible_domain }}'

  root: '/path/to/website'

inventory_nginx_site2:

  default: False

  name: 'othersite.(( ansible_domain }}'

  root: '/other/site'

And my nginx role operates on lists of hashes, so in inventory I can do this:

nginx_servers:

  - '{{ nginx_server_default }}'

  - '{{ inventory_nginx_site1 }}'

  - '{{ inventory_nginx_site2 }}'

This way I can use default server defined by a role for given host, and add additional websites as subdomains of that host (or I could replace default website entirely, of I want to, either by omitting or rewriting 'nginx_server_default' hash (it is defined in nginx/defaults/main.yml). I can also define 'nginx_servers' multiple times, for example in other roles vars/ directory, and make nginx manage their websites (I do that with phpMyAdmin, GitLab, ownCloud and others - you can look in that roles for example usage).

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/3cc833cf-97f3-4fb5-8ab2-db9090dcf29a%40googlegroups.com.

[Serge van Ginderachter]

If you enable the hash_merge behaviour, you can (ab)use hashes:

group_vars/chocolate::

sauces:

  chocolate:

     - name : chocolate

       value : yummy

group_vars/strawberry:

sauces:

  strawberry:

    - name : strawberry

      value : even_yummier

And then iterate over sauces.values()

(You'd probably need to use with_flattened in this case)

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/3cc833cf-97f3-4fb5-8ab2-db9090dcf29a%40googlegroups.com.

[Michael DeHaan]

Correct, the issue was the OP did not have a hash to start with, but a list.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAEhzMJDvvNr9kRn3mXyGYJxZGZixrqn08zLofAX-8uW277Ut9A%40mail.gmail.com.

[Martin Devlin]

Thanks everyone. I think what I was after was confirmation that appending lists in this fashion wasn't supported and it seems it isn't. Maciej Delmanowski's suggestion looks like a viable workaround though so I'll give it a go.

Apologies for the duplicate post in the first instance, Michael

[Smoothify]

I needed the same thing recently, and wrote an internal module that appends one list to another, similar to set_fact.

However, after I had done so I noticed the union filter for jinja templates:
http://docs.ansible.com/playbooks_variables.html#set-theory-filters

This means you could do probably do the following (assuming sauces is already defined somewhere as an empty list):

group_vars/chocolate includes:

sauces_add:

  -name : chocolate
   value: yummy

sauces: "{{ sauces | union(sauces_add) }}"

or you could even chain the merges:

sauces: "{{ sauces | union(sauces_chocolate) | union(sauces_strawberry) }}"

[Martin Devlin]

I tried this but couldn't get this to work in variables files. I think union is only available in templates.

Serge van Ginderachter's suggestion is the only one I've tried that meets my requirement and it's not ideal as he admits himself.

I thought stacking variables like this would make sense; I can see quite a few uses for it in a hierarchical inventory structure, but the fact that no one has requested it as a feature suggests to me I'm approaching group_vars in the wrong way; I'm coming to it from using hiera in puppet.

[Michael DeHaan]

hiera is brain damage that comes up because Puppet didn't really model groups and such correctly to begin with.

As for your union example above, when you can't get something to work, say how and give examples of what you tried and what you got back.   It's the only really way we can help with questions.

Thanks!

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/099a7d14-1d02-4686-b1db-f2fedb46459a%40googlegroups.com.

[Martin Devlin]

OK, thanks for persevering with this! So in group_vars I have lists like this:

ext_vhosts:

  - name         : site1
    hostname     : "{{ site1_hostname }}"
    ipaddress    : "{{ site1_ip_address }}"
    ssl_cn       : "{{ site1_ssl_hostname }}"
    ssl_key      : site1.key
    ssl_crt      : site1.crt
    ssl_pem      : site1.pem
    apache_dir   : site1
    conf_file    : site1.conf

  - name         : site2
    hostname     : "{{ site2_hostname }}"
    ipaddress    : "{{ site2_ip_address }}"
    ssl_cn       : "{{ site2_ssl_hostname }}"
    ssl_key      : site2.key
    ssl_crt      : site2.crt
    ssl_pem      : site2.pem
    apache_dir   : site2d
    conf_file    : site2d.conf
 
int_vhosts:

  - name         : site3
    hostname     : "{{ site3_hostname }}"
    ipaddress    : "{{ site3_ip_address }}"
    ssl_cn       : "{{ site3_ssl_hostname }}"
    ssl_key      : site3.key
    ssl_crt      : site3.crt
    ssl_pem      : site3.pem
    apache_dir   : site3
    conf_file    : site3.conf

  - name         : site4
    hostname     : "{{ site4_hostname }}"
    ipaddress    : "{{ site4_ip_address }}"
    ssl_cn       : "{{ site4_ssl_hostname }}"
    ssl_key      : site4.key
    ssl_crt      : site4.crt
    ssl_pem      : site4.pem
    apache_dir   : site4d
    conf_file    : site4d.conf
 
etc...

I can use ext_vhosts and int_vhosts just fine, I just can't join them together.

The test case is as simple as:

debug : msg="{{ vhosts }}"

So this in the <role>/defaults/main.yml:

vhosts : []
vhosts : "{{ vhosts|union(ext_vhosts)|union(int_vhosts) }}

Yields an empty list

vhosts        : "{{ vhosts|union(ext_vhosts) }}"
vhosts        : "{{ vhosts|union(int_vhosts) }}""

Yields an empty list

In both cases debug : msg="{{ ext_vhosts }}" shows the expected values.

If I put the same structures in vars/main.yml I get "an unexpected type error occured. Error was unhashable type: 'dict'"

If I put the union statements in the respective group_vars files I get "recursive loop detected in template string {{vhosts|union(ext_vhosts)}}"

I hope it's just a case of putting them in a different file or that there's just a bit of missing syntax somewhere.

[Martin Devlin]

Apologies, the empty list outcomes are when I have left a vhosts : [] knocking about somewhere. The error is in fact the same in all cases:

"recursive loop detected in template string"

[Brian Coca]

​create new vars, not a self referencing one

[Martin Devlin]

Yes I imagine the problem with union is that a list can't union with itself. I could try:

dummy : []

vhosts : "{{ dummy|union(ext_hosts)etc...

[Brian Coca]

I'm not sure why you need an empty list to start, am I missing something?

--
Brian Coca
Stultorum infinitus est numerus
0110000101110010011001010110111000100111011101000010000001111001011011110111010100100000011100110110110101100001011100100111010000100001

Pedo mellon a minno

[Brian Coca]

I would write it this way:

vhosts : "{{ ext_vhosts|union(int_vhosts) }}"
​

[Martin Devlin]

Of course, thanks. I'll give it a go on Monday.

Thanks again everyone for your help.

[Martin Devlin]

I still think having the ability to append to lists is a valid use case for a roles-orientated solution.

Consider the examples of packages or iptables or sudoers permissions. Lists are the obvious way to express these. So:

You have a base role for your organisation specifying common requirements. In addition you have specific requirements for particular roles. Moreover, in some circumstances, you might want nodes to be multi-roled.

The ability to append to lists in the way I've described frees you from having to consider, a priori, which set of these requirements a given task will have to address. The tasks can concentrate simply on applying the requirements, leaving the user free to define them in whatever (possibly dumb) way they see fit.

This is constructive feedback; I really like the product and will choose it over puppet every time from now given the choice.

[Brian Coca]

have you tried vhosts.appen(ext_hosts)?​ i don't know if that works but it would be built in if jinja2 allows it

[Martin Devlin]

No I haven't but that looks the same as you'd do it directly in python so it's certainly worth a try. The original request was about whether this was handled implicitly within ansible.