Error in playbook causes handlers not to run

[nafg]

Hi. I have a playbook which had a task (template) with a handler (restart a service), and another task (copy), which had a typo. When I ran ansible-playbook, it reported the error and did not trigger the handler. However the template was updated. I then fixed the typo and ran ansible-playbook again. The second task now succeeded, however this time there was no change detected in the first task and so again the handler was not run.

I think that should never happen. The service should have been restarted, probably the first time, since the error really had nothing to do with the first task or its handler.

This is with ansible 1.4.

Thanks!

[Michael DeHaan]

Yep, this is a thing, there's no way to force running all of the handlers.   

There's a feature request open for something like --force-handlers that would be super easy to add.

But would you generically want to run all of them in every case?  Might not work for everyone.  Might though.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Anand Buddhdev]

On Friday, 29 November 2013 17:14:37 UTC+1, Michael DeHaan wrote:

Hi Michael,

Yep, this is a thing, there's no way to force running all of the handlers.   

There's a feature request open for something like --force-handlers that would be super easy to add.

But would you generically want to run all of them in every case?  Might not work for everyone.  Might though.

I have also run into this issue, where an error in a playbook will cause ansible to stop, and fail to run any requested handlers, which can leave a daemon running with an old configuration, for example. This kind of breaks ansible's idempotent feature. Instead of --force-handlers, what about if ansible were to maintain a persistent cache of requested handlers (in a file, for example) and apply them at the next run?

In our case, we use ansible in local mode, so we can't always see if a playbook run failed or not, and if ansible kept a cache of handlers, and deleted it only when they were applied, it would solve this problem without requiring a --force-handlers option.

Regards,

Anand

[Michael DeHaan]

Not sure that's an option that fits all use cases.

-- Michael

--

[Anand Buddhdev]

Okay, what about making ansible run any queued up handlers just before exiting, even if it exits due to an error condition in a playbook?

Anand

[Michael DeHaan]

"Okay, what about making ansible run any queued up handlers just before exiting, even if it exits due to an error condition in a playbook?"

Yep, that's what I was thinking of this morning.

I still think we *may* want that behavior to be flag/setting controlled.

[C. Morgan Hamill]

Excerpts from Michael DeHaan's message of 2013-12-01 08:56:27 -0500:

> "Okay, what about making ansible run any queued up handlers just before
> exiting, even if it exits due to an error condition in a playbook?"
>
> Yep, that's what I was thinking of this morning.
>
> I still think we *may* want that behavior to be flag/setting controlled.

Perhaps, alternatively, we could have an additional key for plays called
"cleanup_tasks" or some such that means "run everything here no matter
what happens". Or a key for handlers that specifies a handler as
a "cleanup" handler and thus should run even if the play fails. Or
a "notify_on_failure" or "notify_when" that is used next to a "notify"
and specifies under what conditions the handlers ought to be executed?

I'm just snowballin' here. I'm rappin' wit ya.
--
Morgan Hamill

[Jim Ursetto]

I've experienced two similar issues with handlers not running:

  - a misbehaving host hangs forever during a task, forcing you to kill the playbook run, so no handlers for anyone are ever notified

  - a host's task succeeds but its handler times out (e.g. due to flaky connection), so that handler is never run

Perhaps the queued up handlers might fix the first one, and the --force-handlers (with an optional handler filter?) would be useful for the second.

Since the handlers are not idempotent and I have to kind of baby them anyway, I have resorted to converting them to tasks and either running them unconditionally, or having them depend on something more reliable as in a Makefile-like timestamp dependency.  This isn't ideal, but it beats getting out of sync.

Maybe there is no solution that fits within the design goals of ansible?

[Pierre Beck]

I think the current behavior is lacking, but could be worse. There are many things to consider when executing handlers. When a playbook run aborts, it might have been in the middle of reconfiguring a daemon. A handler which restarts said daemon would put the system out of service, so not executing the handler when something aborted the playbook seems fine to me. But the fact that the handler still needs to run should be recorded somewhere on the target host, in a form that describes when in the playbook run the handler would have been executed normally. Then, repeating the playbook could schedule the handler at the same point and execute if it doesn't fail again.

A storage could be something like "~/.ansible/postponed-handlers/". During role execution, a file named like the role would fill with handler names not yet present in the file. When role execution is finished, all handlers in the file would be executed and deleted line by line. Inexistent handler names (when playbook was modified) would be discarded.