Splitting data, or work, across workers

[Sam Zhang]

Hi folks,

I'm a beginner to Ansible. I'd like to split my data across several hosts. These are EC2 instances that I deployed during an earlier play. I'm not sure whether there's an idiomatic way, or a module, that can handle this.

The easiest thing would be to have a module, say "split_file", that takes a single file and splits it across my hosts.

The next best thing, and much more flexible, would be to have a counter that increments for each host. Assuming I have already split my data into my number of workers, I would like to be able to say within each worker task:

- file: src=data/users-{{host_index}}.csv dest=/mnt/users.csv`.

Then, each worker can process their copy of users.csv with a separate script, that is agnostic to which set of users they have. This counter gives me the flexibility to put all the data on all the nodes, and to partition their tasks at the command stage too. It seems like there must be a way to do this. What do you all suggest?

I have copied half of this question over from my StackOverflow post: http://stackoverflow.com/questions/32929823/partitioning-data-across-hosts-in-ansible-access-index-of-host-in-task

Thanks in advance,
Sam