Accessing the host names in the ansible_hosts file in a loop in Ansbile ?

[Soumya Simanta]

I'm trying to automate zookeeper install using Ansible. I want to add a bunch of lines to my cfg file. Following is an example where there are 5 nodes. zoo1, zoo2, zoo3 etc the hosts that I've in my ansible_hosts file.

I want to implement it out with with_items but cannot figure it out. Any idea how to access the host name in the ansible_hosts file in a loop ?

PS : Cross posting here for better coverage. http://stackoverflow.com/questions/24249492/accessing-the-host-names-in-the-ansible-hosts-file-in-a-loop-in-ansbile

initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888
server.4=zoo4:2888:3888
server.5=zoo5:2888:3888

[Serge van Ginderachter]

Say you put your zookeeper nodes in a group 'zookeeper', you can access/loop them with

with_items: groups['zookeeper']

where {{ item }} will get the hostname as defined in your inventory

HTH,

Serge

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/352c1173-2d5b-474d-a769-30cc1a961909%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Soumya Simanta]

Great ! thanks. 

Now what if I want to put a different value 1,2,3 .... to a file /tmp/zookeeper/myid ? 

The my id file already exists but I want to an auto increment number which is different for each machine? 

Thanks. 

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/u0j-DhqmOmc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAEhzMJAiTwRnNKVirFsuzdouKZOcbF4VyADyoG9z3hFLXzoeew%40mail.gmail.com.

[Serge van Ginderachter]

On 16 June 2014 20:32, Soumya Simanta <soumya....@gmail.com> wrote:

Now what if I want to put a different value 1,2,3 .... to a file /tmp/zookeeper/myid ? 

The my id file already exists but I want to an auto increment number which is different for each machine? 

​sounds like nesting lookup plugins​; anible suppport for that is limited though

have a look at with_nested

[Soumya Simanta]

This is what I've and it work. It's not clean though.  I'm sure there is a better way. Because this work for 4 servers but won't work for a N without me changing the line   - ["1","2","3","4"] 

    - name: Add server.number=hostname:2888:3888 the server configuration to config file  zoo.cfg

      lineinfile: dest={{zk_root_dir}}/zookeeper-3.4.6/conf/zoo.cfg line="server.{{ item.0}}={{ item.1 }}:2888:3888"

      with_together:

        - ["1","2","3","4"] #FIXME - this is a hack that needs to be fixed later

        - groups['zookeeper']

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/u0j-DhqmOmc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAEhzMJAN0rT4gS1BLC5PZoad8vwFomDYU7xWMGKt-C1ATbrpMQ%40mail.gmail.com.

[Soumya Simanta]

Now I've another problem.  

If I want to write the server id to myid file then it writes it 4 times. 

    - name: Add server.number=hostname:2888:3888 the server configuration to config file  zoo.cfg

      lineinfile: dest={{zk_root_dir}}/zookeeper-3.4.6/conf/zoo.cfg line="server.{{ item.0}}={{ item.1 }}:2888:3888"

      lineinfile: dest={{zk_tmp_dir}}/myid line="{{ item.0}}"

      with_together:

        - ["1","2","3","4"] #FIXME - this is a hack that needs to be fixed later

        - groups['zookeeper']

In my /tmp/myid file I get 

1 

2 

3 

4 

but I want to have 1 in host1:myid, 2 in host2:myid, 3 in host3:myid and 4 in host4:myid

Any ideas ? 

[Petros Moisiadis]

On 06/16/2014 10:48 PM, Soumya Simanta wrote:

Now I've another problem.  

If I want to write the server id to myid file then it writes it 4 times. 

    - name: Add server.number=hostname:2888:3888 the server configuration to config file  zoo.cfg

      lineinfile: dest={{zk_root_dir}}/zookeeper-3.4.6/conf/zoo.cfg line="server.{{ item.0}}={{ item.1 }}:2888:3888"

      lineinfile: dest={{zk_tmp_dir}}/myid line="{{ item.0}}"

      with_together:

        - ["1","2","3","4"] #FIXME - this is a hack that needs to be fixed later

        - groups['zookeeper']

In my /tmp/myid file I get 

1 

2 

3 

4 

but I want to have 1 in host1:myid, 2 in host2:myid, 3 in host3:myid and 4 in host4:myid

Any ideas ? 

Separate your tasks keeping the same loop construct for both tasks and at the second task (the one that creates '/tmp/myid') add a conditional to run it only on the host of the current loop iteration:

- name: Add server.number=hostname:2888:3888 the server configuration to config file  zoo.cfg
  lineinfile: dest={{zk_root_dir}}/zookeeper-3.4.6/conf/zoo.cfg line="server.{{ item.0}}={{ item.1 }}:2888:3888"
  with_together:

   - (range(groups['zookeeper']|count)|list)[1:] + [groups['zookeeper']|count]
   - groups['zookeeper']

- name: Create /tmp/myid file

  lineinfile: dest={{zk_tmp_dir}}/myid line="{{ item.0}}"

  when: item.0 == inventory_hostname
  with_together:
    - (range(groups['zookeeper']|count)|list)[1:] + [groups['zookeeper']|count]
    - groups['zookeeper']

As you can see, I have replaced your hack for the id list with another hack, so that your id list is dynamically expanded as you add more hosts in the 'zookeeper' group.

If you don' t like hacks, there is a better way to handle this. Just add a 'server_id' host variable for each host, and then your tasks would be like this:

- name: Add server.number=hostname:2888:3888 the server configuration to config file  zoo.cfg

  lineinfile: dest={{zk_root_dir}}/zookeeper-3.4.6/conf/zoo.cfg line="server.{{ hostvars[item].server_id }}={{ item }}:2888:3888"
  with_together: groups['zookeeper']

- name: Create /tmp/myid file
  lineinfile: dest={{zk_tmp_dir}}/myid line="{{ hostvars[item].server_id }}"
  when: item == inventory_hostname
  with_together: groups['zookeeper']

On Mon, Jun 16, 2014 at 3:38 PM, Soumya Simanta <soumya....@gmail.com> wrote:

This is what I've and it work. It's not clean though.  I'm sure there is a better way. Because this work for 4 servers but won't work for a N without me changing the line   - ["1","2","3","4"] 

    - name: Add server.number=hostname:2888:3888 the server configuration to config file  zoo.cfg

      lineinfile: dest={{zk_root_dir}}/zookeeper-3.4.6/conf/zoo.cfg line="server.{{ item.0}}={{ item.1 }}:2888:3888"

      with_together:

        - ["1","2","3","4"] #FIXME - this is a hack that needs to be fixed later

        - groups['zookeeper']

On Mon, Jun 16, 2014 at 2:34 PM, Serge van Ginderachter <se...@vanginderachter.be> wrote:

On 16 June 2014 20:32, Soumya Simanta <soumya....@gmail.com> wrote:

Now what if I want to put a different value 1,2,3 .... to a file /tmp/zookeeper/myid ? 

The my id file already exists but I want to an auto increment number which is different for each machine? 

​ sounds like nesting lookup plugins​; anible suppport for that is limited though

have a look at with_nested

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/u0j-DhqmOmc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAEhzMJAN0rT4gS1BLC5PZoad8vwFomDYU7xWMGKt-C1ATbrpMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CALrNVjWD_cTBmfXzxc8-7LkXyzcaESBm%2BkfAXgH02BDonGAq%2Bg%40mail.gmail.com.

[Petros Moisiadis]

Error. in the second example, 'with_together' should be 'with_items'.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/539FE6BD.4030007%40yahoo.gr.

[Petros Moisiadis]

Stupid me... If you use a 'server_id' variable you do not need to loop for the second task. Just do:

- name: Create /tmp/myid file

  lineinfile: dest={{zk_tmp_dir}}/myid line="{{ server_id }}"

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/539FE756.1080800%40yahoo.gr.

[Michael DeHaan]

Jinja2 has a special variable called "loop.index" that it seems you can use here, and then it's just the template module.

Keep it simple and don't put extra logic in your playbook.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/539FEC22.1030704%40yahoo.gr.

[Petros Moisiadis]

On 06/17/2014 10:10 PM, Michael DeHaan wrote:

Jinja2 has a special variable called "loop.index" that it seems you can use here, and then it's just the template module.

BTW, accessing the loop index works only in the template module. It does not work in ansible's loops with lookup plugins. This is an inconsistency. It should be possible in both cases. That, as well as back referencing to the iterators in nested loops are things that come natural to people trying to construct loops, but, unfortunately, are not available.

Keep it simple and don't put extra logic in your playbook.

Well, in that particular case, using a "server_id" inventory variable as I suggested is as simple as using "loop.index" and even better, because you can remove, add or reorder hosts in the group without causing a change to the server id of each host.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgwPM2w%3DtoBth7tLMjaVTHiF7YS9_pYbxZ1CC%2B6CYs4MPQ%40mail.gmail.com.

[Michael DeHaan]

" It should be possible in both cases."

No, it shouldn't.   Ansible play books don't use Jinja2 for loops - nor should Jinja2 be used in Ansible for logic.

In this case, the user needs to produce a configuration file, so using a template is exactly what is needed.

Your server_id solution is suboptimal and requires manual maintaince of that file when it could be automatically generated.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53A13A93.5070402%40yahoo.gr.

[Petros Moisiadis]

On 06/18/14 18:50, Michael DeHaan wrote:

" It should be possible in both cases."

No, it shouldn't.   Ansible play books don't use Jinja2 for loops - nor should Jinja2 be used in Ansible for logic.

You say "do not" and "should not" in a quite generalized manner, but you don't give any reason.

In contrast, I give specific reasons for this: consistency in available variables between loops in templates and "with_*" loops, as well as adding more possibilities for the "with_*" loops. As you are expanding your deployments to more use cases (especially in the clustering domain) you will eventually need to construct task loops that allow you to do such basic things as accessing the loop index and back referencing to iterators. Templates will not always help you do what you want if that is to run task loops. Of course, you can always come up with hachish, time consuming, "return to bash/python/whatever" techniques, using one template task to create an one-time script that actually does the loop you want, a second task that runs that script and a third task that removes that script. All this could be avoided by just giving a little more power on "with_*" loops.

In this case, the user needs to produce a configuration file, so using a template is exactly what is needed.

I agree.

Your server_id solution is suboptimal and requires manual maintaince of that file when it could be automatically generated.

Deployments should be data-driven (one of Ansible's basic principles and yours of course) with direct mapping between your data and the desirable setup. Changing server ids in an ad-hoc manner through an indirect manipulation of a list of hosts could be highly dangerous in clustering/distributed environments (e.g. could lead to split brains, lost quorum or other unpredictable inconsistencies). I would not recommend that...

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgxGFLAqVLwh08ht562_iD%3DMQJTw05m71N5UjccA%3DmjGLg%40mail.gmail.com.

[Michael DeHaan]

" As you are expanding your deployments to more use cases (especially in the clustering domain) you will eventually need to construct task loops that allow you to do such basic things as accessing the loop index and back referencing to iterators"

This is also a general statement without a reason, FWIW.

This is very much a niche use case.  Not really interested.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53A1C7E7.7030706%40yahoo.gr.

[Petros Moisiadis]

On 06/20/2014 12:16 AM, Michael DeHaan wrote:

" As you are expanding your deployments to more use cases (especially in the clustering domain) you will eventually need to construct task loops that allow you to do such basic things as accessing the loop index and back referencing to iterators"

This is also a general statement without a reason, FWIW.

I have already given reasons and specific examples. Maybe you have not read my other posts carefully.
I have also made a patch with a commit message that analyzes the case with "with_nested". You rejected that on the grounds that it seemed 'too complex' to you. I opened a discussion at ansible-dev where I proved that this "too complex" argument was in fact incorrect: the syntax to access the iterators is already there and used by all 'with_nested' tasks at the end of the nested loop. You have not participated in that discussion...

This is very much a niche use case.  Not really interested.

The general pattern for accessing loop iterators at any time is this: Run a task loop on a specific host, iterating over data structures for each host in a group of hosts. If you cannot understand that definition, there are clear examples, as I said, that will help you understand.

Moreover, at least 3 other users (beside me) have openly said that they tried the obvious (access the iterators before the end of the nested loop) and came to ask about this in this list:
- https://groups.google.com/forum/#!topic/Ansible-project/M3MpXIRXhbQ
- https://groups.google.com/forum/#!topic/ansible-project/SSKdtFHNs2Y
- https://groups.google.com/forum/#!topic/ansible-project/Fh5CEfP1MYM

Let' s see the facts about my solution:
- Does it allow to construct task loops over complex data that cannot be done otherwise? Yes.
- Have other people tried to make such loops and failed? Yes
- Has anyone suggested a better solution for that usage pattern? No
- Does it introduce a new syntax to judge it as 'too complex' ? No
- Does it introduce a new variable to judge it as 'too complex' ? No
- Does it need implementation? No, there is a PR for it already.
- Are people forced to use it or know about it? No

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgwr9Nr0O6iFCf6rvoCOMgHvXgKDDQO4qogs_Uo6Gty-Vg%40mail.gmail.com.

[Michael DeHaan]

"Maybe you have not read my other posts carefully."

"You have not participated in that discussion..."

I am increasingly tired of the pointed tone in these posts.

Consider this a warning.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53A3F39F.8010508%40yahoo.gr.