Deploy directory hierarchy of templates

[Dick Visser]

Hi

I’m looking to deploy a number of configuration files using the template module. The files are structured in a directory hierarchy that needs to be maintained. So for this purpose I want to use the with_filetree lookup as mentioned on 

https://docs.ansible.com/ansible/latest/plugins/lookup/filetree.html. 

This is (at least) a two step process whereby the directory structure is created first, and the actual templates are done next.

The example also lists a third task that recreates symlinks if one needs that. 

While this probably works fine, I do see an issue when a template is deployed first and then removed from the local directory later. 

Redeployment would not remove this ‘orphan’ file from the target directory which will cause issues with my application. 

So I think a final ‘cleanup’ step would be needed that deletes any files on the target that aren’t in the local directory.

Before I do any wheel reinvention - does someone already have such a task at hand?

Thx

Dick

--

Sent from a mobile device - please excuse the brevity, spelling and punctuation.

[Dick Visser]

OK to answer my own question, it wasn't very trivial, but I got it to
work the way I wanted.
Entire playbook:

---
- name: Template directory structure
hosts: localhost
connection: local
gather_facts: no

vars:
template_src_dir: templates/app_config
template_dest_dir: /tmp/etc/app

tasks:
- name: Create directories
file:
path: "{{ template_dest_dir }}/{{ item.path }}"
state: directory
mode: "{{ item.mode }}"
with_filetree: "{{ template_src_dir }}/"
when: item.state == 'directory'

- name: Template files (explicitly skip directories in order to
use the 'src' attribute)
template:
src: '{{ item.src }}'
dest: "{{ template_dest_dir }}/{{ item.path }}"
mode: '{{ item.mode }}'
with_filetree: "{{ template_src_dir }}/"
when: item.state == 'file'

# Start orphan removal logic
- name: Find all remote content
find:
paths: "{{ template_dest_dir }}"
recurse: yes
file_type: any
register: _remote

- name: Find all local content
find:
paths: "{{ template_src_dir }}"
recurse: yes
file_type: any
register: _local

- set_fact:
# Files means anything other than a directory
_remote_files: "{{ _remote | json_query('files[?!isdir].path')
| map('regex_replace', '^' ~ template_dest_dir ~ '/', '') | list }}"
_remote_dirs: "{{ _remote | json_query('files[?isdir].path') |
map('regex_replace', '^' ~ template_dest_dir ~ '/', '') | list }}"
_local_files: "{{ _local | json_query('files[?!isdir].path') |
map('regex_replace', '^' ~ template_src_dir ~ '/', '') | list }}"
_local_dirs: "{{ _local | json_query('files[?isdir].path') |
map('regex_replace', '^' ~ template_src_dir ~ '/', '') | list }}"

- name: Clean up orphaned files
file:
path: "{{ template_dest_dir }}/{{ item }}"
state: absent
loop: "{{ _remote_files | difference(_local_files) }}"

- name: Clean up orphaned directories
file:
path: "{{ template_dest_dir }}/{{ item }}"
state: absent
loop: "{{ _remote_dirs | difference(_local_dirs) }}"

--
Dick Visser
Trust & Identity Service Operations Manager
GÉANT

[Brian Coca]

I once had a similar problem and I solved it in also a similar way,
then made it into a role https://galaxy.ansible.com/bcoca/tidy

--
----------
Brian Coca

[Dick Visser]

Sorry for resurrecting this old thread.
I shuffled things around a bit lately, and came up with a single
playbook file that can be used to template a directory tree. See
below.

---
# Template a directory structure
# Include this task like this example, supplying at least
# a 'src' and 'dest' variable:
#
# - include_tasks: template_tree.yml
# vars:
# src: "{{ playbook_dir }}/app1/config"
# dest: /etc/app1
#
# Optional variables:
# validate: /usr/bin/chckconfig %s
# cleanup: true

- name: Find all local content

become: false
connection: local
find:
paths: "{{ src }}"
recurse: true
hidden: true
file_type: any
register: _local


- name: Ensure directories are created
file:
path: "{{ dest ~ '/' ~ ( item.path | relpath(src)) }}"
state: directory
mode: "{{ item.mode }}"
loop: "{{ _local | json_query('files[?isdir]') }}"
loop_control:
label: "{{ item.path | relpath(src) }}"


- name: Ensure files are templated
template:
src: '{{ item.path }}'
dest: "{{ dest ~ '/' ~ ( item.path | relpath(src)) }}"
mode: '{{ item.mode }}'
validate: "{{ validate | default(omit) }}"
loop: "{{ _local | json_query('files[?!isdir]') }}"
loop_control:
label: "{{ item.path | relpath(src) }}"


- name: Remove orphan logic
block:

- name: Find all remote content
find:

paths: "{{ dest }}"
recurse: true
file_type: any
hidden: true
register: _remote

- name: Clean up orphaned files/directories
file:
path: "{{ dest }}/{{ item }}"
state: absent
loop: "{{ _remote | json_query('files[].path') | map('relpath',
dest) | list |
difference( _local | json_query('files[].path') |
map('relpath', src) | list) }}"

when: cleanup|default(false)|bool




I'm using this in
https://github.com/dnmvisser/ansible_role_simplesamlphp/blob/master/tasks/template_tree.yml

> --
> You received this message because you are subscribed to the Google Groups "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CACVha7fReFrLSBnEOKUv9Ah4U9eYVkmRu_kN9hRqw2iUuS6aKQ%40mail.gmail.com.