Tags not propagated to dynamically included tasks: Workaround?

[Frank Thommen]

Dear all,

since "recently" (we realized just now), tags applied in role's main.yml
task, like:

/role/myrole/tasks/main.yml:

- name: My Task
include_tasks: "mytask.{{ansible_osfamily}}.yml"
tags:
- mytask-only
- always

are not propagated to the included task any more. It still works fine
for statically included tasks. This seems to be the the now documented
way of attribute inheritance since ansible 2.5 and it probably came to
us with a recent `yum upgrade` which updated ansible from 2.4 to 2.6 in
one big step.

However this is a major pain for us: We are extensively using unified,
role-spanning[*] tags on the level of the the main tasks to be able to
selectively roll out our configurations. The new behaviour means, that
we have to manually add hundreds(!) of "tag" statements to each
individual task in all the included taskfiles.

Is there a way to achieve the former behaviour (tag inheritance to
dynamic includes) other than framing each task.yml with a block, which
introduces redundancy and error-proneness?


Personally I find this new behaviour suboptimal regarding maintenance,
intuitive understanding of playbooks and scalability. All things, that
I appreciated with ansible so far.

Cheers
frank



[*] i.e. we have e.g. the same tag "sysconfig" applied to an included
taskfile for the role "common" and for the role "hw-server" and for the
role "webserver", allowing us to run the complete system configuration
with --tags=sysconfig

[Kai Stian Olstad]

On Friday, 24 August 2018 15.47.39 CEST Frank Thommen wrote:
> since "recently" (we realized just now), tags applied in role's main.yml
> task, like:
>
> /role/myrole/tasks/main.yml:
>
> - name: My Task
> include_tasks: "mytask.{{ansible_osfamily}}.yml"
> tags:
> - mytask-only
> - always
>
> are not propagated to the included task any more.

<skip />

> However this is a major pain for us: We are extensively using unified,
> role-spanning[*] tags on the level of the the main tasks to be able to
> selectively roll out our configurations. The new behaviour means, that
> we have to manually add hundreds(!) of "tag" statements to each
> individual task in all the included taskfiles.

You don't need to add it to every task, you can use a block: around them all and set the tag on the block.

You can also wrap the include_tasks in a block, and set the tags on the block.

- block:

- name: My Task
include_tasks: "mytask.{{ansible_osfamily}}.yml"
tags:
- mytask-only
- always

> Is there a way to achieve the former behaviour (tag inheritance to
> dynamic includes) other than framing each task.yml with a block, which
> introduces redundancy and error-proneness?

Only the workaround mention above, but you can always downgrade.
There are some discussion don't remember if it was in an issue or on this list about a option or something like that to tell if it should propagate or not.


--
Kai Stian Olstad

[Frank Thommen]

Thanks a lot Kai,

Very nice solution. I myself experimendet with a construction like:

- name: "My Task"
block:
- import_tasks: mytask.RedHat.yml
when: distro == "RedHat"
- import_tasks: mytask.CentOS.yml
when: distro == "CentOS"
- import_tasks: mytask.Suse.yml
when: distro == "Suse"
- import_tasks: mytask.Debian.yml
when: distro == "Debian"
tags:
- mytask-only

which makes for bad reading and bad maintainability. Your idea with the
block is elegant and well maintainable while keeping all the dynamic and
flexibility.

I realize blocks are more versatile than I thought ;-)

f.

[stan.v...@exlhub.com]

This is an interesting post that touches upon the topic: https://medium.com/opsops/ansible-2-5-delegate-to-and-include-role-20cd7e67008e

The way I look at it, Ansible team has chosen a path of keeping the product competitive vs. maintaining backward compatibility at all costs. The Best Practices recommend avoiding Tags all together, as well as other traditional features that are static in nature and aren't considered in dynamic Includes. This (supposedly) allows writing better playbooks. Yes, there is a potentially considerable effort re-writing old ones that don't comply with the new model. On the other hand, with the speed of the industry changes, since the time the old books were written, an average team would probably switch to another tool or technology all together as part of the cycle, so keeping Ansible as a valuable option for future cycles pretty much means that the archaic way the old playbooks were written in has to change.

On Friday, August 24, 2018 at 11:17:27 PM UTC+8, Frank Thommen wrote:

Thanks a lot Kai,

[Brian Coca]

A few of things:

 - `include:` still exists, yes we are planning to eventually deprecate as it has mixed static/dynamic behaviour and it is not easy to know which you will get

 - `include_role`/`include_tasks` were created to represent the 'dynamic' parts of the existing `include`, but they have been a work in progress, we do think we have almost finalized them now and hope to mark them as 'stableinterface' soon

 - `import_role`/`import_tasks`/`import_playbook` are the 'static' parts, this helps disambiguate the `include:` behaviors and make them explicit.

 - the distinctions should be easy now import_X has all keywords set as 'inheritance', while include_X have all keywords apply to it, but not inherited. This still leaves things that make no sense like `delegate_to` or `become` on includes, but we are planning on documenting and warning on those in the future.

 - `include_role`/`include_tasks` have gotten the `apply` keyword (in 2.7) that allows for inheritance, in older versions you can use a `block` as shown in previous posts. We hope that this rounds out all the functionality needed for these features.

We do try to maintain backwards compatibility, but in this case that means `include` as `include_tasks` and `include_roles` were 'new' and 'in progress' as we figured out how people are using them and how we think it best suits everyone's needs, so backwards compatibility on those was not as important as getting the feature set 'right'.  

User feedback has been crucial for this and part of it was that users were displeased with things changing even if they liked the changes. We are trying to figure out better ways to communicate the state of a feature and adjust expectations as not everything we release is complete, perfect and immutable.

Thanks for your feedback and your patience,

--------------

Brian Coca

[Kai Stian Olstad]

On Monday, 27 August 2018 16.59.03 CEST Brian Coca wrote:
> - `include:` still exists, yes we are planning to eventually deprecate as
> it has mixed static/dynamic behaviour and it is not easy to know which you
> will get
>
> - `include_role`/`include_tasks` were created to represent the 'dynamic'
> parts of the existing `include`, but they have been a work in progress, we
> do think we have almost finalized them now and hope to mark them as
> 'stableinterface' soon

I think this might be the source of all frustration, include was marked deprecated before the new feature was stable.

The Ansible community is so big that you would have gotten a lot of feedback without marking include deprecated at that point.

> We do try to maintain backwards compatibility, but in this case that means
> `include` as `include_tasks` and `include_roles` were 'new' and 'in
> progress' as we figured out how people are using them and how we think it
> best suits everyone's needs, so backwards compatibility on those was not as
> important as getting the feature set 'right'.

I think your would have gotten this feedback without marking include deprecated.
Just mention the new feature in the release notes and say comment accepted would have gone a long way.

> User feedback has been crucial for this and part of it was that users were
> displeased with things changing even if they liked the changes. We are
> trying to figure out better ways to communicate the state of a feature and
> adjust expectations as not everything we release is complete, perfect and
> immutable.

I'm one of does that likes the changes, had no issue with it. I only used when on include, and the change was very welcome.
But I understand the painfulness some user have experience.
When you see a deprecation warning you naturally want that message to do a way, but non of the new solution could do that out of the box, some tweaking was necessary.

--
Kai Stian Olstad

[Brian Coca]

We reverted the deprecation on include, the feedback did make it clear
we had made a mistake at the time.
We have revised our deprecation policy to trail the stable substitute
vs always being immediate.

--
----------
Brian Coca

[Michael Ströder]

On 8/24/18 4:31 PM, Kai Stian Olstad wrote:
> You can also wrap the include_tasks in a block, and set the tags on the block.
>
> - block:
> - name: My Task
> include_tasks: "mytask.{{ansible_osfamily}}.yml"
> tags:
> - mytask-only
> - always

Given the volatility of include handling in a pretty short period my
question is:
Is the solution above guaranteed to work for the next few years?

Ciao, Michael.