replace module: how to replace all matching files

[Andrei Baban]

Hi, 

I have a variables.yml file in which I store variables in this format:

patterns:

  - file: file_1.xml

    regexp: 17701

    replace: 17706

  - file: file_2.xml

    regexp: 17701

    replace: 17706

  - file: file_3.xml

    regexp: 17701

    replace: 17706

then, in my playbook I first import these variables:

    - name: Read pattern replaces from file and save them as variables

      include_vars: ~/path_to_file/variables.yml

and then do replaces on these files like this:

    - name: perform pattern replaces for files

      local_action:

        module: replace

        follow: yes

        path: '~/path_to_files/{{ item.file }}'

        regexp: '{{ item.regexp }}'

        replace: '{{ item.replace }}'

      with_items: 

        - "{{ patterns }}"

All works well, but I need to be able to specify files with regexp in the variables.yml file. Something like below:

patterns:

  - file: file_*.xml

    regexp: 17701

    replace: 17706

  - file: other_file.xml

    regexp: a

    replace: b

This will fail as replace module will output that 'file_*.xml' cannot be found

I can do this by using shell module and sed command but I'd really like to stick to using the correct modules instead of falling back to bash. Do you have any suggestion on how to achieve something like this?

thanks!

  

[Kai Stian Olstad]

On Tuesday, 14 November 2017 17.16.15 CET Andrei Baban wrote:
> All works well, but I need to be able to specify files with regexp in the
> variables.yml file. Something like below:
>
> patterns:
> - file: file_*.xml
> regexp: 17701
> replace: 17706
> - file: other_file.xml
> regexp: a
> replace: b
>
> This will fail as replace module will output that 'file_*.xml' cannot be
> found
>
> I can do this by using shell module and sed command but I'd really like to
> stick to using the correct modules instead of falling back to bash. Do you
> have any suggestion on how to achieve something like this?

To this you would need to use the find module to find the files matching the filename regexp/glob, then feed the filename result into the replace module.


--
Kai Stian Olstad

[Andrei Baban]

thanks Kai for your answer! 

the issue with find module would be that it would work fine if there would be only one type of file with one replace in it. But I store in the variables file multiple type of files that each have different patterns to search for and replace. I would need in pseudocode a for each syntax that would run over both find and replace, something like

for each type of file {

    find all matching files

    replace all resective patterns relative only to this file

}

hope this makes sense :)

[Kai Stian Olstad]

On Wednesday, 15 November 2017 16.08.39 CET Andrei Baban wrote:
> thanks Kai for your answer!
>
> the issue with find module would be that it would work fine if there would
> be only one type of file with one replace in it. But I store in the
> variables file multiple type of files that each have different patterns to
> search for and replace. I would need in pseudocode a for each syntax that
> would run over both find and replace, something like
>
> for each type of file {
> find all matching files
> replace all resective patterns relative only to this file
> }
>
> hope this makes sense :)

You have this functionality in Ansible by using include/include_task, with_item/with_dict and loop_control.


--
Kai Stian Olstad

[Andrei Baban]

thanks! 

I'll research this a bit more, as I still don't see a clear way yet on how to condition the second loop to run only on a subset of properties read from the variable file; but the loop_control is a good start. 

[Kai Stian Olstad]

On Friday, 17 November 2017 09.03.55 CET Andrei Baban wrote:
> thanks!
>
> I'll research this a bit more, as I still don't see a clear way yet on how
> to condition the second loop to run only on a subset of properties read
> from the variable file; but the loop_control is a good start.

This is not 100% correct code, just an idea on how it could be solved with the information you provided in the fist mail.

vars:

patterns:
- file: file_*.xml
regexp: 17701
replace: 17706
- file: other_file.xml
regexp: a
replace: b

tasks:
- include: change.yml
with_items: '{{ patterns }}'
loop_control:
loop_var: outer_item


change.yml
---
- find:
paths: /your/path
patterns: '{{ outer_item.file }}'
register: filelist

- name: perform pattern replaces for files
local_action:
module: replace
follow: yes

path: '{{ item.path }}'
regexp: '{{ outer_item.regexp }}'
replace: '{{ outer_item.replace }}'
with_items: "{{ filelist.files }}"


--
Kai Stian Olstad

[Andrei Baban]

I think this is exactly what I've needed! Thanks a lot for your time and patience explaining this. I'll test it on Monday and reverse to proper module usage instead of shell + sed.

thanks!