loops and variables

[Makimoto Marakatti]

Hi all

Could someone lend me a hand to figure looping over variables?

I've got a number of hosts which are grouped in the inventory quite strictly in a rather traditional way:

dev-group1
  hostA
  hostB
tst-group1
  hostC
  hostD
pre-group1
  hostE
  hostF
pro-group1
  hostG
  hostH
dev-group2
  hostI
  hostJ
tst-group2
  hostK
  hostL
and so on...

I'm about to replace/template a great number of files in them and I'd like to have local backups first ;)
The only requirement is that the files have to be fetched to an already established structure that mimics the inventory:

/backup/dev-group1/
/backup/tst-group1/
etc...

So I made an attempt to loop over each group:

$ ansible-playbook fetch_from_server.yml -sK --extra-vars "@fetch_loop_vars.yml" -v

fetch_loop_vars.yml looks like this:

---
file: '/root/.bash_profile'
my_groups: ['0dm-sap', 'dev-sap', 'tst-sap', 'pre-sap', 'pro-sap']

And my fetch_from_server.yml is this:

---
- hosts: '{{my_groups}}'
  sudo: True
  gather_facts: no
  tasks:
    - name: fetch the file from the server
      fetch: src={{file}} dest=/backup/{{my_groups}}/{{inventory_hostname}}

I get this result when running:

ok: [hostA] => {"changed": false, "dest": "/backup/[dev-group1,/hostA/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}
ok: [hostB] => {"changed": false, "dest": "/backup/[dev-group1,/hostB/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}
ok: [hostD] => {"changed": false, "dest": "/backup/[dev-group1,/hostC/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}
ok: [hostC] => {"changed": false, "dest": "/backup/[dev-group1,/hostD/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}

And I'd like to get:

ok: [hostA] => {"changed": false, "dest": "/backup/dev-group1/hostA/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}
ok: [hostB] => {"changed": false, "dest": "/backup/dev-group1/hostB/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}
ok: [hostD] => {"changed": false, "dest": "/backup/tst-group1/hostC/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}
ok: [hostC] => {"changed": false, "dest": "/backup/tst-group1/hostD/root/.bash_profile", "file": "/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078"}

So the problems are:
- It's not iterating over {{my_groups}}. Every host is placed under the first defined group: dev-group1
- It's actually writing the dir of the group as "[dev-group1," which of course is not what I need.

I'm sure this is a trivial issue for those fluent in python, which obviously is not my case.

Any help appreciated
Thanks!

[Abhijit Menon-Sen]

At 2014-05-22 05:21:35 -0700, maki...@gmail.com wrote:
>
> fetch_loop_vars.yml looks like this:
> ---
> file: '/root/.bash_profile'
> my_groups: ['0dm-sap', 'dev-sap', 'tst-sap', 'pre-sap', 'pro-sap']
>
> And my fetch_from_server.yml is this:
> ---
> - hosts: '{{my_groups}}'
> sudo: True
> gather_facts: no
> tasks:
> - name: fetch the file from the server
> fetch: src={{file}} dest=/backup/{{my_groups}}/{{inventory_hostname}}

This task isn't actually using a loop in the Ansible sense. You need to
provide a list to iterate over using with_items: or similar. For a more
detailed explanation, see http://docs.ansible.com/playbooks_loops.html
(it has lots of examples).

That said, I don't know the best way to iterate over the hosts in a
subset of all your groups in quite the way you describe. Perhaps the
examples on the following page will prove helpful:

http://docs.ansible.com/playbooks_variables.html#magic-variables-and-how-to-access-information-about-other-hosts

-- ams

[Abhijit Menon-Sen]

At 2014-05-22 05:21:35 -0700, maki...@gmail.com wrote:
>

> So I made an attempt to loop over each group:

I'm sorry, my earlier answer was short-sighted. I started thinking about
loops because your Subject said "loop", but you were right to not use an
Ansible loop in the first place—at least not over groups/hosts.

As I understand it, trying to write a single task that loops over groups
and hosts isn't the way to do things in Ansible. One should write a task
that fetches the desired files, and run it against the desired hosts by
using the existing host-selection mechanism, as described in
http://docs.ansible.com/intro_patterns.html

So you could do something like:

- name: fetch files from the server
fetch: src={{ item }}
dest=/backup/{{ group_names[0] }}/{{ inventory_hostname }}/{{ item | basename }}
with_items:
- /root/.bash_profile
- /some/other/file

Note that group_names is a list variable that contains the name of all
groups that the current host is in. If your hosts aren't in multiple
groups, this should not matter to you. If they are, then the backups
will be stored only in the directory of the first group by this task.

Hope this helps.

-- ams

[Makimoto Marakatti]

Thanks for the insightful answer. I'm definitely going to try this and will write here how it goes

[Makimoto Marakatti]

Well it does work... but for the 'wrong' groups.

Let me explain:
There's another layer of groups in the inventory to define in which datacentre a particular host is.
So in a group like dev-group1 we might have 2 machines in datacentre-A and other 2 hosts in datacentre-B.
In this case I'm not interested in where a box is. I need to refer to it by it's role. (Not the ansible role, but what it does :) )

So that is why I had this arbitrary list of groups read from a variable. And that is still what I would need.
I'm thinking to make a play to define those vars and including another one so I can use the "with_items" construct.

If someone has come across something like this before, I would love to hear about it!




We have a number of defined groups which we need

[Brian Coca]

you can try just rewriting my_groups this way:

my_groups: "0dm-sap:dev-sap:tst-sap:pre-sap:pro-sap"

[Makimoto Marakatti]

Brian:
Unfortunately that yields this:

changed: [hostA] => {"changed": true, "dest": "/backup/0dm-sap:dev-sap:tst-sap:pre-sap:pro-sap/hostA/hostA/root/.bash_profile", "md5sum": "bab9333347e752b87add49020919a078", "remote_md5sum": "bab9333347e752b87add49020919a078"}

I also failed to make this work declaring vars and including another play.

So I guess it's back to the usual: roles.
But for this particular task, that seems so inelegant...

thanks to all

[Brian Coca]

yeah, didn't realize you were reusing it for the task also, my 'fix' would work only for hosts: entry.​

[Michael DeHaan]

Or make a group that contains those groups as children, so it's cleaner :)

On Thu, May 22, 2014 at 11:51 AM, Brian Coca <bria...@gmail.com> wrote:

you can try just rewriting my_groups this way:

my_groups: "0dm-sap:dev-sap:tst-sap:pre-sap:pro-sap"

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHsycxjO%2B3RNXMgr87f-8MiMJcTvstymiQe%2Buy5Ybsw81Nw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Michael DeHaan]

You could make a backup role and do something like this:

- hosts: group1

  roles:

     - { role: backup, path: 'group1' }

- hosts: group2

  roles:

     - { role: backup, path: 'group2' }

A host can be in lots of groups, so you shouldn't rely on the group_names variable to tell you what you are looping over.

For instance, a host could be in a group based on purpose and another based on geography.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/73e4e65b-e10e-45f0-a7df-0bfe4423aa78%40googlegroups.com.

[Makimoto Marakatti]

> A host can be in lots of groups, so you shouldn't rely on the group_names variable to tell you what you are looping over.

Well that was why I was using "my_groups". I do have a lot of groups in the inventory...

Your suggestion seems to be like something that could work. I'll try to adapt that and report back.

Thanks!

[Dmitry Makovey]

Why not "with_nested"?

as per ams:

    - name: fetch files from the server 

      fetch: src={{ item }} 

        dest=/backup/{{ item[1] }}/{{ inventory_hostname }}/{{ item[0] | basename }} 

      with_nested: 

        - [ /root/.bash_profile, /some/other/file ]

        - group_names

better yet - provide list of files as a variable.

[Makimoto Marakatti]

Just seen this today.
Will give it a try soon...

Thanks!