Parsing csv file and running shell command

[alonso...@gmail.com]

Hello All,

I am trying to use ansible to read values from a csv file (source and destination) and then run commands using the shell module, but I am having no success. Please assist .

The playbook:

---

  - name: Read Users

    gather_facts: True

    hosts: localhost

    tasks:

    - name: Read users

      read_csv:

        path: users.csv

        delimiter: ","

      register: s3list

    - name: Running command

      command: echo "aws s3 cp {{ s3.source }}" {{ s3.destination }} -- recursive --acl-parameters ---profile prod'

      loop: "{{ s3list.list }}"

      loop_control:

        loop_var: s3

the csv file contents:

source,destination
S3://mybucket/puppy/test.jpg,S3://mybucket2/trying/
S3://mybucket/puppy/ds.jpg,S3://mybucket3/trying/
S3://mybucket/puppy/as.jpg,S3://mybucket4/trying/
S3://mybucket/puppy/was.jpg,S3://mybucket5/trying/
S3://mybucket/puppy/qaes.jpg,S3://mybucket6/trying/

When I run the playbook I am getting the following
TASK [Running command] *********************************************************************************************************************
changed: [localhost] => (item={'source': 'S3://mybucket/puppy/test.jpg', 'destination': 'S3://mybucket2/trying/'})
changed: [localhost] => (item={'source': 'S3://mybucket/puppy/ds.jpg', 'destination': 'S3://mybucket3/trying/'})
changed: [localhost] => (item={'source': 'S3://mybucket/puppy/as.jpg', 'destination': 'S3://mybucket4/trying/'})
changed: [localhost] => (item={'source': 'S3://mybucket/puppy/was.jpg', 'destination': 'S3://mybucket5/trying/'})
changed: [localhost] => (item={'source': 'S3://mybucket/puppy/qaes.jpg', 'destination': 'S3://mybucket6/trying/'})

PLAY RECAP *********************************************************************************************************************************
localhost                  : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

I would like to run the aws command, but it doesnt seem to work 

[Vladimir Botka]

You're not running the *aws* command. You're testing the *aws* command
with *echo*. Actually, this is a good idea to see the commands before
you run them. Register the results and take a look at *stdout*.

In the module *command*, there is a couple of syntax problems
in the run-string. To make the syntax both simpler and easier to read
use the *folded style*. See
https://yaml.org/spec/1.2.2/#813-folded-style

The tasks below

- command: >-
echo
aws s3 cp
{{ item.source }}
{{ item.destination }}
--recursive --acl parameters --profile prod
loop: "{{ s3list.list }}"
register: out
- debug:
msg: "{{ out.results|map(attribute='stdout')|list }}"

should give you what you want

msg:
- aws s3 cp S3://mybucket/puppy/test.jpg S3://mybucket2/trying/
--recursive --acl parameters --profile prod
- aws s3 cp S3://mybucket/puppy/ds.jpg S3://mybucket3/trying/
--recursive --acl parameters --profile prod
- aws s3 cp S3://mybucket/puppy/as.jpg S3://mybucket4/trying/
--recursive --acl parameters --profile prod
- aws s3 cp S3://mybucket/puppy/was.jpg S3://mybucket5/trying/
--recursive --acl parameters --profile prod
- aws s3 cp S3://mybucket/puppy/qaes.jpg S3://mybucket6/trying/
--recursive --acl parameters --profile prod

If this is what you want remove *echo* and run the *aws* command with
the option *--dryrun*

- command: >-
aws s3 cp
{{ item.source }}
{{ item.destination }}
--dryrun --recursive --acl parameters --profile prod
loop: "{{ s3list.list }}"
register: out
- debug:
var: out

If everything is alright copy the files

- command: >-
aws s3 cp
{{ item.source }}
{{ item.destination }}
--recursive --acl parameters --profile prod
loop: "{{ s3list.list }}"
register: out
- debug:
var: out

Notes:

* You can remove the option *--recursive* as long as you copy single
files
* You'll have to replace *parameters* with actual values. See
https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html


--
Vladimir Botka

[alonso...@gmail.com]

Thank you every much, that worked, although I am not certain why echo with shell module wouldnt work , but thank you!