How to automatically detect changes to files managed by Salt State ?

[Bob Moshai]

Hi,

Is there a way to setup salt to automatically detect changes to files managed using Salt states and update the minions.

For e.g. In my service.sls, I have:
/etc/service.conf:
  file:
    - managed
    - source: salt://service.conf

If the service.conf is updated on the Salt master, do I have to manually (or using cron job or salt scheduler) invoke highstate or can Salt handle this using some file system monitoring feature.

Thanks,

Bob Moshai

[Dmitry Malinovsky]

Hi Bob.

I can't say is this feature available. But I'd rather say 'no' for this idea. I think, it would be very dangerous to run highstate (or just state.sls) on each change - this change can break things like nginx very silently, instead of running highstate explicitly.

[Pranav Aggarwal]

One way is, you could write some python code to keep check on your file change and use Slat Python Client to call the salt-minions.

[Colton Myers]

This feature is not built into salt.  That said, it would be pretty easy to roll your own solution.  THAT being said, I wouldn't recommend it, for the same reasons Dmitry mentioned:  automating a highstate would be dangerous if you save an intermediate change that's not ready for primetime.

--

Colton Myers

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[David Anderson]

At the moment, the closest you can get is probably to set up the
scheduler in your minions to either run a highstate or a specific state
at an interval. The minions will then see that the file.managed
salt://service.conf was changed on the master.

http://docs.saltstack.com/topics/jobs/schedule.html
--
Dave

[Les Mikesell]

On Tue, Oct 1, 2013 at 1:02 PM, David Anderson <da...@dubkat.com> wrote:
> At the moment, the closest you can get is probably to set up the scheduler
> in your minions to either run a highstate or a specific state at an
> interval. The minions will then see that the file.managed
> salt://service.conf was changed on the master.

Has anyone done something to version-control managed files? I've
always thought it would be nice to have a system that tracked the
previous states while making changes - perhaps making each node have a
branch against the master copy in a version control system so you only
had to store deltas for the nodes/versions.

--
Les Mikesell
lesmi...@gmail.com

[Tim O'Guin]

You mean like the Git and Mercurial backends? I haven't used the new Mercurial backend from 0.17, but I've managed my states and other files using the gitfs backend. 0.17 has added support for using gitfs for pillar data, too.

[Les Mikesell]

On Tue, Oct 1, 2013 at 2:46 PM, Tim O'Guin <timo...@gmail.com> wrote:
> You mean like the Git and Mercurial backends? I haven't used the new
> Mercurial backend from 0.17, but I've managed my states and other files
> using the gitfs backend. 0.17 has added support for using gitfs for pillar
> data, too.
>

Not quite. I mean besides versioning what you push to a minion,
commit what was previously there to a branch before each change for a
possible rollback and to be able to use the VCS tools to compare
differences over time and/or between hosts. If no changes ever
happen besides what is pushed to the node, the master history might be
enough, but in the real world programs and OS's update themselves,
admins make helpful quick-fixes, etc., etc. and you want to know what
really happened, not just what was supposed to have happened.

And by the way, is there likely to ever be support for subversion?

--
Les Mikesell
lesmi...@gmail.com

[Tim O'Guin]

On Tue, Oct 1, 2013 at 3:02 PM, Les Mikesell <lesmi...@gmail.com> wrote:

On Tue, Oct 1, 2013 at 2:46 PM, Tim O'Guin <timo...@gmail.com> wrote:
> You mean like the Git and Mercurial backends? I haven't used the new
> Mercurial backend from 0.17, but I've managed my states and other files
> using the gitfs backend. 0.17 has added support for using gitfs for pillar
> data, too.
>

Not quite.  I mean besides versioning what you push to a minion,
commit what was previously there to a branch before each change for a
possible rollback and to be able to use the VCS tools to compare
differences over time and/or between hosts.   If no changes ever
happen besides what is pushed to the node, the master history might be
enough, but in the real world programs and OS's update themselves,
admins make helpful quick-fixes, etc., etc. and you want to know what
really happened, not just what was supposed to have happened.

Ah gotcha, like taking and committing a snapshot before applying a state? 

The way we're doing that for our production websites that get deployed from Mercurial (not with Salt) is any changes that get made directly on production get committed to a hotfix branch and pushed to another repo for review, which gets merged back into the main repo and then into the dev and staging environments.

I agree it'd be nice to have something like this for salt, but I have no idea what it would take to implement. Ideally everything goes through source control, but, as you've said, that doesn't always happen.

[Les Mikesell]

On Tue, Oct 1, 2013 at 3:11 PM, Tim O'Guin <timo...@gmail.com> wrote:
> >>
>> Not quite. I mean besides versioning what you push to a minion,
>> commit what was previously there to a branch before each change for a
>> possible rollback and to be able to use the VCS tools to compare
>> differences over time and/or between hosts. If no changes ever
>> happen besides what is pushed to the node, the master history might be
>> enough, but in the real world programs and OS's update themselves,
>> admins make helpful quick-fixes, etc., etc. and you want to know what
>> really happened, not just what was supposed to have happened.
>
>
> Ah gotcha, like taking and committing a snapshot before applying a state?

Yes, but in a central VCS that would have tools to easily show
differences between different systems as well as what changed on a
single host.

> The way we're doing that for our production websites that get deployed from
> Mercurial (not with Salt) is any changes that get made directly on
> production get committed to a hotfix branch and pushed to another repo for
> review, which gets merged back into the main repo and then into the dev and
> staging environments.
>
> I agree it'd be nice to have something like this for salt, but I have no
> idea what it would take to implement. Ideally everything goes through source
> control, but, as you've said, that doesn't always happen.

For content that is strictly your own, pushing it through the VCS is
probably enough - especially if your tools do enough logging to track
back any deployed change to the relevant source change and the person
that approved it for production. But things get messy with OS and
application packages that tend to drag along their own default
configurations and conflict with your local changes during updates.
Linux distributions at least do a half-hearted attempt at abstracting
local settings under /etc/sysconfig/ but it is far from perfect.

--
Les Mikesell
lesmi...@gmail.com

[Steve Butterworth]

On Tuesday, 1 October 2013 15:42:09 UTC-4, LesMikesell wrote:
>  Has anyone done something to version-control managed files?  I've
>  always thought it would be nice to have a system that tracked the
>  previous states while making changes - perhaps making each node have a
>  branch against the master copy in a version control system so you only
>  had to store deltas for the nodes/versions.

If you really want to catch random changes to host configuration that haven't been passed through a central configuration system, you need a host-based solution like *etckeeper*.

Since before experimenting with Salt, I've been using etckeeper to track configuration file changes with Git. It isn't a complete solution, but for the case of files in /etc, I love having the ability to check on changes. (As long as they are in place long enough to get caught by the daily snapshot commit, or at the time of any package installations/upgrades.) It is also wonderful to be able to test a configuration script, check on the changes with "git status" and then decide to roll back with "git reset --hard", or push it to the side with "git stash".)

You could easily build a central repository of these changes by pulling from each host to a central repo. You could even set up commit hooks to auto-push to a central repo, or trigger a central pull.

(I can certainly imagine a model with a branch per host: the ultimate in space efficiency and inter-host diff-testing heaven, but outside the etckeeper model.)

[Les Mikesell]

On Tue, Oct 1, 2013 at 3:47 PM, Steve Butterworth
<steven.bu...@gmail.com> wrote:
> On Tuesday, 1 October 2013 15:42:09 UTC-4, LesMikesell wrote:
>> Has anyone done something to version-control managed files? I've
>> always thought it would be nice to have a system that tracked the
>> previous states while making changes - perhaps making each node have a
>> branch against the master copy in a version control system so you only
>> had to store deltas for the nodes/versions.
>
> If you really want to catch random changes to host configuration that
> haven't been passed through a central configuration system, you need a
> host-based solution like *etckeeper*.

But that's only a part of the picture. I want to know how this host
has diverged from a master image state too, with a quick way to see
what is currently different between any two similarly-configured
hosts.

> You could easily build a central repository of these changes by pulling from
> each host to a central repo. You could even set up commit hooks to auto-push
> to a central repo, or trigger a central pull.
>
> (I can certainly imagine a model with a branch per host: the ultimate in
> space efficiency and inter-host diff-testing heaven, but outside the
> etckeeper model.)

I think it needs to all start with a common master with the hosts
committed as branches to make VCS tools work the way you would want.
But this seems so obvious that someone must already have the framework
written.

--
Les Mikesell
lesmi...@gmail.com

[Dmitry Golubenko]

В Втр, 01/10/2013 в 14:42 -0500, Les Mikesell пишет:

> On Tue, Oct 1, 2013 at 1:02 PM, David Anderson <da...@dubkat.com> wrote:
> > At the moment, the closest you can get is probably to set up the scheduler
> > in your minions to either run a highstate or a specific state at an
> > interval. The minions will then see that the file.managed
> > salt://service.conf was changed on the master.
>
>
> Has anyone done something to version-control managed files? I've
> always thought it would be nice to have a system that tracked the
> previous states

I use etckeeper with plugin(etckeeper-zypp-plugin for suse) that commit
changes before/after installing/removing packages plus states like this:

etckeeper_commit:
cmd.run:
- name: 'git status && git add . && git commit -a -m "saltstack
run"'
- cwd: /etc
- require:
- file: /etc/etckeeper/etckeeper.conf
- pkg: etckeeper
- onlyif: '(cd /etc; exec /usr/bin/test `git status --porcelain | wc
-l` -ne 0 )'
- order: last

for commiting changes made by salt.