Include a task for each element of a list

[Stéphane Dupille]

Hi!

I'd like to manage users on my servers. Basically, I want 3 variables for each group of servers (may be overrided for a specific host) containing the users I want to create :

users: basic users

sudo_users: users in the sudo group

disabled_users: users disabled (or deleted).

So in the common role, I have a task like this :

- include: create_sudo_user.yml user={{item}}

  with_items: sudo_users

- name: echo

  shell: echo {{ item }}

  with_items: sudo_users

sudo_users is a list defined in a group_vars file. The echo task is here for debugging purpose.

The included task is simply:

---

- name: create user {{ user }}

  shell: echo {{ user }}

Here is the result of that playbook:

PLAY [all] ******************************************************************** 

GATHERING FACTS *************************************************************** 

ok: [isonoe.cry.pto]

TASK: [create user sudo_users] ************************************************ 

changed: [isonoe.cry.pto] => (item=sudo_users)

TASK: [echo] ****************************************************************** 

changed: [isonoe.cry.pto] => (item=dupille)

PLAY RECAP ******************************************************************** 

isonoe.cry.pto             : ok=3    changed=2    unreachable=0    failed=0   

Instead of getting the content of the sudo_users var in the task create user, I get the name of the variable itself. Why that variable is not evaluated?

If I update my task to add a constant array, everything works well:

- include: create_sudo_user.yml user={{item}}

  with_items: [a,b,c]

 

- name: echo

  shell: echo {{ item }}

  with_items: sudo_users

PLAY [all] ******************************************************************** 

GATHERING FACTS *************************************************************** 

ok: [isonoe.cry.pto]

TASK: [create user a] ********************************************************* 

changed: [isonoe.cry.pto] => (item=a)

TASK: [create user b] ********************************************************* 

changed: [isonoe.cry.pto] => (item=b)

TASK: [create user c] ********************************************************* 

changed: [isonoe.cry.pto] => (item=c)

TASK: [echo] ****************************************************************** 

changed: [isonoe.cry.pto] => (item=dupille)

PLAY RECAP ******************************************************************** 

isonoe.cry.pto             : ok=5    changed=4    unreachable=0    failed=0   

How can I force the evaluation of my variable ?

Thanks,

[Christopher O'Connell]

Assuming that your list of users is defined in as a list in a vars file, then something like this has worked for me before.

accounts.yml:
- accounts:
  - name: one
    list: [ a ]
  - name: two
    list: [a, b, c]
  - name: three
    list: [ c ]

playbook.yml:
  - include: setup_accounts.yml
    with_items: ${accounts}

setup_accounts.yml:
  - task: Do something for {{ item.name }}
    action: {{ item.name }} works here
  - task: Do something for {{ item.name }} lists
    action: {{ item }} is an item in the list from the vars file
    with_items: ${item.list}

As I asked in another thread, I'm not sure this is a good way to set this up, but it does seem to work.

Presumably, if you setup a vars file with an entry like:

- my_list:
  - value1
  - value2
  - value3

Then simply using {{ item }} in the included file will produce 'value1', 'value2', 'value3'.

All the best,

~ Christopher

[Stéphane Dupille]

Hello!

I defined my accounts list as is, in a vars_file:

sudo_users: [test1,test2,test3]

The problem is that the with_item statement in include tasks doesn't behave the same way as in other tasks.

In any task, the with_item is "evaluated", so I can get the content of the variable. But for include task, the with_item part is not "evaluated", and I can only put some constant value.

So, when I do something like that:

- include: create_sudo_user.yml user={{item}}

  with_items: ${ sudo_users }

I get this error: ERROR: with_items expects a list

If I do

- include: create_sudo_user.yml user={{item}}

  with_items: sudo_users

I get in the included file the value "sudo_users" instead of the content of the list.

In that case, if instead of an include task I do a shell task, it works well.

It seems to me that's a bug: with_items should behave the same way for any task. What do you think?

Oh, for the record, I'm using the current git version, fetched today.

$ ansible --version

ansible 1.3

Many thanks for your help.

[Michael DeHaan]

I see you are using include combined with a with_items.  Don't do that!  :)

This will solve most of your problems.

It was not implemented well originally and I should have never taken the patch for it -- people try to use it with inventory variables and that simply cannot work because of the way the system works.

It's much better to loop inside the task itself, use with_items inside there, and it's fine to pass a list to a role variable if you like to achieve the same level of parameterization.

include + with_items is likely to be removed completely by 1.5.   Please take appropriate steps to not rely on it.

Meanwhile, you're also using old style variables.

with_items: sudo_users

would be enough.

--Michael

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Serge van Ginderachter]

On 3 September 2013 22:53, Michael DeHaan <mic...@ansibleworks.com> wrote:

It was not implemented well originally and I should have never taken the patch for it -- people try to use it with inventory variables and that simply cannot work because of the way the system works.

It's much better to loop inside the task itself, use with_items inside there, and it's fine to pass a list to a role variable if you like to achieve the same level of parameterization.

include + with_items is likely to be removed completely by 1.5.   Please take appropriate steps to not rely on it.

​Whilst I fully understand the problem with include + with_items, the only places I use it, is when I nest loops: ​a double with_items so to speak, where the outer loop doesn't get variables from inventory of course.

Any plans or options to extend or support something similar if include + with_items gets removed? I know about with_nested, not sure if that covers all my use cases.

Serge

[Michael DeHaan]

Let me know where you think with_nested wouldn't work.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[C. Morgan Hamill]

On Tue, September 3, 2013 4:53 pm, Michael DeHaan wrote:
> include + with_items is likely to be removed completely by 1.5. Please
> take appropriate steps to not rely on it.

Is there any reason an include + with_items couldn't be translated to an
include, with every included task having the with_items applied
individually?

Sort of like it works now with include + when? The only issue I can think
of is how to handle the case where an included task also has a with_items.

Thoughts?
--
Morgan Hamill

[Michael DeHaan]

Morgan,

*MAYBE*.   That's a good idea, but it would also have to upgrade any "with_items" it found to "with_nested" calls, and that's confusing because then you have a {{ item.0 }} and {{ item.1 }} to content with.

Better to loop explicitly inside the task.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[C. Morgan Hamill]

On Tue, September 3, 2013 6:41 pm, Michael DeHaan wrote:
> *MAYBE*. That's a good idea, but it would also have to upgrade any
> "with_items" it found to "with_nested" calls, and that's confusing because
> then you have a {{ item.0 }} and {{ item.1 }} to content with.
>
> Better to loop explicitly inside the task.

Right, it's definitely a tricky one.

Would a patch which made is "do the right thing" be accepted, assuming it
wasn't a god-awful hack, or would you rather leave things as they are?
--
Morgan Hamill

[Stéphane Dupille]

Hello,

groumpf.

Looping over an include is, in my opinion, a must-have feature.

What I wanted to do: for each user create a group, create a user, send a ssh-key, and send config files for each user. So I have 4 tasks, and each one for a single user. Yes, I can loop for each task, but it's a little bit more painful.

If that cannot be achieved using a with_items, a construct using a jinja loop may be definitely useful. Is there a way to preprocess playbooks in the jinja templating system? I may be wrong but it seems to me easier to do.

If I cannot loop over an include, I have an other issue. I may have (or not have) in my roles/common/files/config/<username> directory some files I would like to send to my servers (each user may have different config files, a different shell, ...). Does that kind of construct work?

- name: deploy config files

  sudo: yes

  copy: src={{ item }} dest="/home/{{ item }}/" owner={{ item }} mode={{ 600 }}

  with_fileglob:

    - configfiles/{{ item }}/*

  with_items: sudo_users

  ignore_errors: yes

How can I loop both on a list and some files? Can I mix with_items and with_fileglob?

Thanks,

[Michael DeHaan]

I'm afraid it would be a hack, because of other types of iterators.

--
Morgan Hamill

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Michael DeHaan]

"What I wanted to do: for each user create a group, create a user, send a ssh-key, and send config files for each user."

You use with_nested.

[Serge van Ginderachter]

On 3 September 2013 23:28, Michael DeHaan <mic...@ansibleworks.com> wrote:

Let me know where you think with_nested wouldn't work.

​I reviewed the scripts where I use include+with_items, and the problem exists where I combine a with_item loop + with_fileglob, which is also the use case Stéphane mentioned.

​As discussed per irc:

This could be solved - I think - if we could get the output of a lookup plugin as a complex var, a list, and not a flattened string, on a with_ construct:

  vars:

    users:

    - foo

    - bar

  tasks:

  - debug: 'msg="{{ item.0, item.1 }}"'

    with_nested:

    - users

    - lookup('fileglob', 'files/*')

​

Need to further think on implementation.

Serge​

[Michael DeHaan]

As discussed on IRC, I like this suggestion, though we would need to make sure it was implemented for "with_items" and the various other appropriate "with_" statements as well.   

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Christopher O'Connell]

I would like to second the suggestion of allowing lookup plugins to return complex types. It would greatly simplify building reusable play books.

All the best,

~ Christopher

[Serge van Ginderachter]

fyi, feature ticket filed: https://github.com/ansible/ansible/issues/4060