Ansible enhancement request: persistent variables?

[er...@tibco.com]

I'm looking for an enhancement to Ansible that I'm conceiving of as "persistent variables." Specifically, when one part of my script does something that registers a variable, I'm typically using that registration to affect whether or not operations are performed afterwards. However, scripts can fail, and then I've lost that prior state that something happened. Running the script again will typically leave the registered variable in a different state, and the operations that triggered off of the registered variable may not be performed, even though they should be.

For a work-around, for example, looking at packages going to be updated on the system. If the pending package updates include, say, an update to the database software, then my script pseudo code looks like this:

# check whether the DB software is going to be updated, record to a file

# update database

# check if file exists indicating DB software update - flag DB updated

# if db updated, run DB migration tool

# restart database server

# remove flag file indicating DB software being upgraded

-----------------------

Or a similar case:

# check whether apache web server going to be updated, write a file that indicates apache update

# update all packages

# check whether file exists indicating apache update

# if apache updated - optionally rewrite Apache config

# if apache updated - restart apache

# remove flag file indicating apache update

------------------------

When I initially wrote my scripts, I recorded the state of whether or not the DB software or Apache was being updated in a variable.

However, ansible scripts fail. For example, "update all packages" could update Apache, but then fail for any number of reasons. So I want to capture the state of a potential update in a file, so that if Ansible steps fail, the state that the software has been updated has been captured, so that the next time the script runs, it still does the right thing and restarts Apache, for example. I could restart Apache every time, but that is potentially disruptive to clients, so I don't want to do that except if required.

All of which leads me to suggest two enhancements:

1) Persisted state variables - as you can see from the scripts above, I have to write a file saving state (only if update is needed), test that file later - saved into a variable, and then delete the file that holds the state. Instead, if I could conditionally "register" a value for a persistent variable, and then upon successful completion of a script, Ansible itself cleans up all the persistent variables, then I don't need to have steps for: writing to a file, checking that file, and removing that file. This will simplify handling of scenarios where state needs to persist between steps in an Ansible script.

2) Assuming I can get persisted state variables, it would also be useful to enhance the package management tools (portage, yum, apt, etc...) to record persistent state about which packages were updated by an update. Only when the script has run successfully to completion does that variable, and all the packages updated, actually get removed. Now I can write scripts that check whether an update has happened in this run of a script, or any previously unsuccessful run.

Perhaps there's a better way to accomplish some of the above with existing Ansible tasks, but I've not found it.

Eric.

[Ben Friedman]

I'm not sure there's any way to persist state between program executions except by saving that state offline like you are doing.

That being said, I think the goal of ansible is to write your play in a way that sets the state of a system accordingly. For instance, use a handler that only gets invoked when a task is changed. If apache hasn't been changed in any way, the 'restart apache' handler doesn't run.

If you're storing state offline, you'll have to write your play to check that data store for state. You might as well write the play in a way that checks the target state before taking action, if needed.

[er...@tibco.com]

Thanks for the reply!

On Tuesday, December 20, 2016 at 4:45:18 PM UTC-8, Ben Friedman wrote:

I'm not sure there's any way to persist state between program executions except by saving that state offline like you are doing.

Yes, I recognize that. That's why I labeled this an enhancement request, and why I brought the discussion here, rather than just posting a bug report. I'm guessing either other people have run into the same problem and found better solutions, or have the same need of an enhancement.

 

That being said, I think the goal of ansible is to write your play in a way that sets the state of a system accordingly. For instance, use a handler that only gets invoked when a task is changed. If apache hasn't been changed in any way, the 'restart apache' handler doesn't run.

Yes, and I'm tossing in a notion that part of the "system state" that ansible stores is persistent variables whose values persist until a playbook has run to successful completion. Since Ansible cannot be "transactional" in its operation, registering variables always has failure modes. The smaller the range of those failure modes, the better off the ecosystem will be.

Some of Ansible's operations (such as a service restart, or command operations) may leave the target system in exactly the same configured / operational state, but they do have side-effects (like dropping connections, momentarily). Those side effects may be worth avoiding....

 

If you're storing state offline, you'll have to write your play to check that data store for state. You might as well write the play in a way that checks the target state before taking action, if needed.

As I indicated in my examples, that's exactly what I do. However, that makes my playbooks more complicated, more error prone, and harder to understand. And since I don't think I'm the only one with this problem, it strikes me that it could work well as an enhancement to Ansible itself.

[Ben Friedman]

Gotcha! My apologies for assuming you aren't already doing what you gotta do.

I'd like to know if something like this could be added.

What I've been doing is using other tools like rundeck and jenkins to orchestrate my plays and 'roll back' thinks where possible by running other plays.

It does get messy.

[Dick Davies]

My worry about that sort of approach is it assumes the local playbook
directory is authoritative - in our case several people have access to run
our playbooks, so that would get messy very quickly.

Keeping state on the managed hosts would be a better idea, and in that
case you could keep 'lock files' around in /var that are created at
the beginning
of a piece of work, and then removed once everything is updated successfully.

If those scripts are facts , you can use most of Ansibles existing machinery
for free.


On 20 December 2016 at 19:20, eric via Ansible Project

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/bd5e22a2-6fb4-4923-ad06-541de3c98a3d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Brian Coca]

Keeping state is something we have actively avoided in Ansible, it brings up a whole slew of problems, most of the time more than it would solve.

Using block/rescue/always you can specifically address certain operations which you believe need this extra state being tracked (template + conditional include_vars), this is not something we want to build into Ansible itself.

----------

Brian Coca