Dynamic reusability of a playbook — include_tasks, include_role, import_playbook … Mission impossible?

[Stas Fomin]

Hello all.

Consider following sample project:
  https://github.com/belonesox/ansible_dynamic_role_howto

With tree:
.                                                                                                                                                                           
├── playbook.yml                                                                                                                                                            
├── run-playbook                                                                                                                                                            
└── somedir                                                                                                                                                                 
    └── dynamically_defined_dir                                                                                                                                             
        ├── do.yml                                                                                                                                                          
        ├── roles                                                                                                                                                           
        │   ├── role1                                                                                                                                                       
        │   ├── role2                                                                                                                                                       
        │   │   ├── defaults                                                                                                                                                
        │   │   │   └── main.yml                                                                                                                                            
        │   │   ├── tasks                                                                                                                                                   
        │   │   │   └── main.yml                                                                                                                                            
        │   │   └── templates                                                                                                                                               
        │   │       └── sometemplate.j2                                                                                                                                     
        │   └── role3                                                                                                                                                       
        ├── run-nested-playbook                                                                                                                                             
        ├── templates                                                                                                                                                       
        └── vars             

Here we have here:
* A working playbook "do.yml" in "dynamically_defined_dir" (can be run by "run-nested-playbook" shell script).
* A playbook "playbook.yml" that tried to use "nested" playbook "do.yml", trying to redefine some of defaults of nested playbook.
* Playbook "playbook.yml" should be run  ("run-playbook") with some external parameters like this:
   ansible-playbook -vvv -i 'localhost,' -c local playbook.yml  --extra-vars "external_extra_var=somedir"

Expected (wanted) result for "run-playbook"
* Get contents of sometemplate.j2 with replacing default variables to some variables/parameters from  "playbook.yml"
* "Playbook.yml" should be without hardcoding paths to nested playbook (should be calculated somehow from "--extra-vars"). Better by "set_fact" some variable, and reuse it.

----
What I have tried. ↓↓↓↓

First of all: "run as shell: ansible_playbook xxx xxx " is not an option
because then I completely lost execution progress of nested playbook.

----


"import_playbook"

→
   import_playbook: "{{external_extra_var}}/dynamically_defined_dir/do.yml" 
* OK, can use external extra vars
* But it cannot use calculated variables:  
  - hosts: 127.0.0.1
    connection: local
    tasks:
      - set_fact: wtf_dir="{{external_extra_var}}/dynamically_defined_dir"
      - include_tasks: "{{external_extra_var}}/dynamically_defined_dir/roles/role2/tasks/main.yml" 
    - import_playbook: "{{wtf_dir}}/do.yml"
We got "ERROR! 'wtf_dir' is undefined"

Yes, from  https://docs.ansible.com/ansible/2.4/playbooks_reuse_includes.html + http://docs.ansible.com/ansible/latest/playbooks_reuse.html
we have
"All import* statements are pre-processed at the time playbooks are parsed.
 All include* statements are processed as they encountered during the execution of the playbook."

So it works as expected, and here no "include_playbook" twin for "import_playbook" :(

OK. Lets drop "playbook" reusability, returns to "roles" reusability.

 ---
 - hosts: 127.0.0.1
   connection: local
   tasks:
    - set_fact: wtf_dir="{{external_extra_var}}/dynamically_defined_dir"
    - include_role:
        name: "{{wtf_dir}}/roles/role2"
      static: no

Same problem → "ERROR! 'wtf_dir' is undefined" 

Probably this is a bug https://github.com/ansible/ansible/issues/19438

----
OK. Lets try to reuse nested playbook roles by "include_tasks".

  - hosts: 127.0.0.1
    connection: local
    tasks:
      - set_fact: wtf_dir="{{external_extra_var}}/dynamically_defined_dir"
      - include_tasks: "{{wtf_dir}}/roles/role2/tasks/main.yml" 

Wow! No more "ERROR! 'wtf_dir' is undefined"… but "template" lookup not works as it should:
  "An unhandled exception occurred while running the lookup plugin 'template'. … the template file sometemplate.j2 could not be found for the lookup"   

I see, that it look for the template  
in
* ./template
* ./somedir/dynamically_defined_dir/template
* ./somedir/dynamically_defined_dir/roles/role2/tasks/template
but not in right place:
* ./somedir/dynamically_defined_dir/roles/role2/template

--------------------

So the questions is:
* Did I miss some another way to archieve my goal?
* May be I do something wrong obviously… Any workaround?
* "include_role" behavior — bug or feature?
* "include_playbook" absense — should we ask for implementation?

Thanks all in advance,
  Sincerely, Stas

P.S.  ansible-playbook 2.4.1.0

[J Hawkesworth]

There are further changes to dynamic inclusion of tasks in ansible 2.5, so probably worth while testing with latest code rather 2.4.x versions now.

see https://docs.ansible.com/ansible/devel/porting_guide_2.5.html

and https://groups.google.com/forum/#!topic/ansible-project/GsJ7GjUhI_g regarding testing out 2.5 alpha build.

Hope this helps,

Jon