Help wanted: "Including" playbooks for a "Playbook of Playbooks"

[Kedwin Chen]

I am trying to "include" a lot of other playbooks in a "playbook of playbooks" (I am aware, and have tried to use, the import_playbook and include modules to no avail), and am trying to apply the ignore_errors: yes to the imported/included playbook; since I am testing, and not all hosts are available, I would like to ignore the connection errors.

How can I write my main playbook (all-programs.yml) such that I can run through playbooks/program[A-Z].yml while ignoring errors should they occur?

Scenario:

I am running Ansible v. 2.7.10, with the following layout

.
├── group_vars
│   └── superGroup
│       ├── vars.yml
│       └── vault.yml
├── hosts
├── playbooks
│   ├── all-programs.yml # This is the playbook that I am struggling to write
│   ├── programA.yml
│   ├── programB.yml
│   ├── programC.yml
│   ...
│
├── roles
│   ├── programA
│   │   └── tasks
│   │       └── main.yml
│   ├── programB
│   │   └── tasks
│   │       └── main.yml
│   ├── programC
│   │   └── tasks
│   │       └── main.yml
...

Here is what I have tried in playbooks/all-programs.yml so far (here, ... means the end of the YAML file):

# Attempt 1
---
- name: Run playbook for ProgramA
  import_playbook: "{{ playbook_dir }}/programX.yml"
  ignore_errors: yes 

- name: Run playbook for ProgramB
  import_playbook: "{{ playbook_dir }}/programY.yml"
  ignore_errors: yes 

...


######

# Attempt 2
---    
- name: Run playbook for ProgramA
  include: "{{ playbook_dir }}/programA.yml"

- name: Run playbook for ProgramB
  include: "{{ playbook_dir }}/programB.yml"
...


######

# Attempt 3
---    
- name: Run playbook for ProgramA
  hosts: localhost
  tasks:
    - include: "{{ playbook_dir }}/programA.yml"

- name: Run playbook for ProgramB
  hosts: localhost
  tasks:
    - include: "{{ playbook_dir }}/programB.yml"
...


######

# Attempt 4
---    
- name: Run all playbooks in a block
  hosts: localhost
  tasks:
      - name: Try ProgramA playbook
        block:
            - name: "playbook for programA"
              import_playbook: "{{ playbooks_dir }}/programA.yml"
        rescue:
            - debug:
                msg: "programA manipulation playbook failed"
        ignore_errors: yes

      - name: Try ProgramB playbook
        block:
            - name: "playbook for programB"
              import_playbook: "{{ playbooks_dir }}/programB.yml"
        rescue:
            - debug:
                msg: "no programB"
        ignore_errors: yes
...



######

# Attempt 5: Copy contents of program[A-Z].yml playbooks into master playbook all-programs.yml
---

- name: Run role of ProgramA for required hosts
  hosts: hostA,hostC,groupB
  gather_facts: no
  strategy: free
  roles:
    - programA

- name: Run role of ProgramB for required hosts
  hosts: hostD,groupW
  gather_facts: no
  strategy: free
  roles:
    - programB
...

Running with the following command line for each attempt above yields:

ansible-playbook --ask-vault-pass -i /path/to/hosts -e "playbook_dir=/path/to/playbooks" /path/to/playbooks/all-programs.yml

# Attempts 1-3:

PLAY [Run playbook for Program[A|B] *********************

TASK [(subtask of other_role)] **************************
fatal: [hostA-Z]: UNREACHABLE! => {"changed: false", ...}

NO MORE HOSTS LEFT **************************************
    to retry, use: --limit @/path/to/retry

#########################################################

# Attempt 4:

PLAY [Run all playbooks in a block] *********************

TASK [Gathering Facts] **********************************
ok: [localhost]

TASK [playbook for programA] ****************************
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 0}
...ignoring

TASK [playbook for programB] ****************************
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 0}
...ignoring

From above, what might be the module failure being referred to?

General layout/schema of playbooks/program[A-Z].yml (here, ... means the end of the YAML file):

# playbooks/programA.yml
---
- name: Run role of ProgramA for required hosts
  hosts: hostA,hostC,groupB
  gather_facts: no
  strategy: free
  roles:
    - programA
...

# playbooks/programB.yml
---
- name: Run role of ProgramY for required hosts
  hosts: hostD,groupW
  gather_facts: no
  strategy: free
  roles:
    - programB
...

General layout/schema of roles/program[A-Z]/tasks/main.yml (here, ... means the end of the YAML file):

---
- name: Role for ProgramA
  include_role:
    name: other_role
  vars:
    var1: something
    var2: somethingElse
...

Thank you for taking your time to read this post.

Any assistance would be appreciated.

[Karl Auer]

This sounds like an extraordinarily bad idea. Could you tell us what you are doing this *for*?

Regards, K.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6776ba24-a467-40d3-ac90-90d1fae41539%40googlegroups.com.

--

Karl Auer

Email  : ka...@2pisoftware.com
Website: http://2pisoftware.com

GPG/PGP : 301B 1F4E 624D AD99 242C 7A68 EC24 7113 E854 4A4E
Previous: 958A 2647 6C44 D376 3D63 86A5 FFB2 20BC 0257 5816

[Dick Visser]

If the problem is that some hosts are not available, I would address that by simply limiting the playbook to not talk to those hosts:

https://docs.ansible.com/ansible/latest/user_guide/intro_patterns.html

In general the “Try something and ignore failure” approach will mask all errors, and will prevent your setup from being idempotent. 

There is a reason for the failure. Find out what that is, then use that as condition to run the task.

Dick 

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6776ba24-a467-40d3-ac90-90d1fae41539%40googlegroups.com.

--

Sent from a mobile device - please excuse the brevity, spelling and punctuation.

[Kedwin Chen]

I am trying to do an automated software deployment, with each system requiring a different set of software.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6776ba24-a467-40d3-ac90-90d1fae41539%40googlegroups.com.

[Mauricio Tavares]

Why not define a bunch of variables for each host/whatever and
then have the different playbooks be called, include things, and do
other things depending on what those variables tell them?

> To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/881df92d-c231-44e3-bc98-53b171d87263%40googlegroups.com.

[Kedwin Chen]

I am not sure what you mean by "prevent my setup from being idempotent" -- assuming that I have (probably) written my roles to check if the changes have not already been applied before the tasks to apply changes run, should the failure of one playbook's hosts not just remove those unreachable hosts from future plays, and fall through and continue running?

As far as reason for failure, is that not just the fact that the host does not exist in this environment?

My first thought would be to write a conditional to only "include" the playbook if the host returns a successful pong, but Ansible marks unreachable hosts as fatal errors, and once the first included playbook runs out of hosts, the entire playbook of playbooks aborts.

I could solve this by limiting the playbook to not talk to the hosts, but QA runs the playbook *exactly* as I run it in test, and so if I limit it in test, QA limits it in their inspection, and the inspection will fail because not all hosts were touched.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6776ba24-a467-40d3-ac90-90d1fae41539%40googlegroups.com.

[Josh Smift]

We want to do this as a check that all of our hosts are in synch; we have a playbook per system type, and we don't want the check to fail if e.g. the only host of a given type is down (which would thus be a 100% failure of that playbook).

We do it like this:

- import_playbook: thing1.yml
- import_playbook: meta-clear-host-errors.yml


- import_playbook: thing2.yml
- import_playbook: meta-clear-host-errors.yml

[* etc *]

...where meta-clear-host-errors.yml is this:

---


# Include this playbook in playbooks that include other playbooks,
# if you don't want errors for some hosts in one playbook
# to disable those hosts in later playbooks.


- name: Clear hosts with errors from the failed host lists
  hosts: all
  gather_facts: False
  tasks: [ meta: clear_host_errors ]

Although now that I think about it, this may *not* prevent the 100% failures problem, because I think we still run into that from time to time. This may just help deal with other sorts of failures, like if a host is in multiple playbooks, and some tasks in one fail, we don't want that host to be missing when the other playbook runs.