Ability for a host to declare which roles should be applied to it?

[Philip Kauffman]

I'd like to start a discussion on the ability to define roles to `import_role` from within host_vars. Instead of *only* defining those roles statically in a playbook.

My goal is to be able to give an inventory_host the ability to define any [extra] roles it should import [on top of what is expressed in the playbook].

The inspiration for this comes from an extensive use of Puppet and Hiera in production.

### What I'd like to get out of this discussion:
1. How weird is this? / "correctness" / best practices
2. Are there other methods I had not considered?
3. Am I just running into a fundamental limit of the tool?

## My investigation and methods on how a host could define its own roles

This is what I was dealing with in my `site.yml` before my investigation:
```
- hosts:
  - hostname1
  user: deployuser
  become: yes
  gather_facts: yes
  roles:
  - common
  - router
  - route53
  - cloudflare
  vars_files:
  - "./vars/specificfilename.yml"

- hosts:
  - hostname2
  user: deployuser
  become: yes
  gather_facts: yes
  roles:
  - common
  - router
  - route53
  vars_files:
  - "./vars/specificfilename.yml"

- hosts:
  - hostname3
  user: deployuser
  become: yes
  gather_facts: yes
  roles:
  - common
  - wireguard_client
  - zeus
  vars_files:
  - "./vars/specificfilename.yml"

...
... more hosts with roles applied in a similar manner
... all with slightly different role sets
...
```

Lots of repeated lines where most of the time I actually just need to adjust the role list.


### Before we begin:
1. Lets assume that I have good reasons for what I want to do, even if that ends up not being the case. I'd like to explore this to see if it is a bad idea.
2. Inventory organization: I am aware I can create groups of hosts and apply roles to those groups. Unfortunately, I have a few roles where that would be useful, but most hosts will get a set of their own unique roles.
3. Directory structure:
```
./site.yml
./roles/include-roles/
./roles/render-role-import/
./roles/proxmox/ # installs proxmox-ve and used as an example role
./rendered/{{ inventory_hostname }}/ # created by role:render-role-import
./host_vars/{{ inventory_hostname }}/custom_roles
        # contents
        ---
        roles:
        - role1
        - role2
        - role3
        - etc
```

### My solutions so far:

#### Create a role whose only job it is, is to `include_role` from host_vars/<hostname>/custom_roles.

```
site.yml
---
- hosts: all
  roles:
  - include-roles
```

```
roles/include-roles/tasks/main.yml
---
- name: host_vars defined roles
  include_role:
    name: "{{ r }}"
  loop: "{{ roles }}"
  loop_control:
    loop_var: r
```

This is reasonably effective, but I lose the ability to use tags I've set in the roles I include.

This is super annoying since I specifically setup my roles like this to make it easier while building and debugging.

```
roles/proxmox/tasks/main.yml
---
- include: hosts.yml
  tags:
  - proxmox
  - proxmox-hosts

- include: repo.yml
  tags:
  - proxmox
  - proxmox-repo

- include: install.yml
  tags:
  - proxmox
  - proxmox-install

- include: web-redirect.yml
  tags:
  - proxmox
  - proxmox-web-redirect

- include: nag.yml
  tags:
  - proxmox
  - proxmox-nag
```

```
ansible-playbook site.yml -l whale --tags=proxmox

PLAY [whale] ***************************************************************************************
skipping: no hosts matched

PLAY [whale] ***************************************************************************************
skipping: no hosts matched

PLAY RECAP ****************************************************************************************
```
This is because `include_role` is dynamic and import_role is static. Those includes have not happened at the time you specify the tags.


Ok, fine. Let's try to use `import_role` instead. Spoiler: you _cannot_ use loops.

So we continue and use loops anyways... but not in the same way as above.


#### Render a task file for the specific host and include that in the playbook.

```
roles/render-role-import/tasks/main.yml
---
- name: create rendered directory
  file:
    path: "{{ item }}"
    state: directory
  with_items:
  - ./rendered
  - "./rendered/{{ inventory_hostname }}"
  delegate_to: localhost

- name: render role import task file
  template:
    src: tasks.yml.j2
    dest: "./rendered/{{ inventory_hostname }}/roles.yml"
    mode: '0644'
  delegate_to: localhost
```

```
roles/render-role-import/templates/tasks.yml.j2
---

{% for r in roles %}
- import_role:
    name: {{ r }}

{% endfor  %}
```

Which results in the following file being created:
```
rendered/whale/roles.yml
---

- import_role:
    name: common

- import_role:
    name: proxmox

```

Now my `site.yml` looks like this:
```
site.yml
---
- hosts: all
  connection: local
  roles:
  - render-role-import
  tags:
  - always

- hosts: all
  user: deployuser
  become: yes
  gather_facts: yes
  tasks:
  - include: "./rendered/{{ inventory_hostname }}/roles.yml"
```

This sort of works. You still cannot use `--list-tags`. But the role tags (from <rolename>/main.yml) are available and can be used.

```
$ ansible-playbook site.yml --tags=proxmox-install

PLAY [whale] **************************************************************************************
... removed for clarity ...
TASK [include] ************************************************************************************
included: /home/myuser/ansible/cloud/rendered/whale/roles.yml for whale
... removed for clarity ...
```

It then proceeds to run the roles that are imported in `rendered/whale/roles.yml` or the specific tag you specified.

### Conclusion
Overall the rendered task file that imports all the roles does most everything I need, but it feels incorrect. Like I abused the tool into working how I wanted it to work and not how it was intended to be used. Is there some Ansible-ism that I'm not aware of?

[flowerysong]

Set `tags: always` on your include_role task, so that it always runs.

On Wednesday, December 8, 2021 at 12:31:16 PM UTC-5 phi...@kauffman.me wrote:

I'd like to start a discussion on the ability to define roles to `import_role` from within host_vars. Instead of *only* defining those roles statically in a playbook.

My goal is to be able to give an inventory_host the ability to define any [extra] roles it should import [on top of what is expressed in the playbook].

```
roles/include-roles/tasks/main.yml
---
- name: host_vars defined roles
  include_role:
    name: "{{ r }}"
  loop: "{{ roles }}"
  loop_control:
    loop_var: r
```

This is reasonably effective, but I lose the ability to use tags I've set in the roles I include.

[Philip Kauffman]

Thanks. However, I"m not actually using that role.

My main question is if rendering 'import_role' task files follows best practices or if there is an alternative to what I'm doing I'm not aware of.

[flowerysong]

No, that kind of hack is about as far from best practices as you can get. As I said in my previous message, the alternative that you were not aware of was using `include_role` with the `always` tag.

[Philip Kauffman]

Thanks for the clarification. I understand what you mean now and in retrospect seems quiet obvious. I'm glad I can do away with the rendering; I felt gross writing it.

I guess there is no official means of a host defining its own roles?