conditional restart of service

[Jason Lavoie]

I'm trying to have a exec dependency on a service object that would keep
it from being restarted if the exec fails. Unfortunately, the service
gets refreshed regardless whenever the exec is run, failure or not.

I've tried various combinations of subscribe/require/notify, but can't
find an elegant way to not restart the service when the dependency
fails. I must be missing something obvious.

We have a few needs for this model, here's a (simplified) example of one:

class dhcp::server {
package { dhcp3-server: ensure => installed }

service { dhcp3-server:
ensure => running,
hasstatus => true,
require => [ Package["dhcp3-server"], Exec["check-dhcpd-conf"] ],
}

exec { "check-dhcpd-conf":
command => "/usr/sbin/dhcpd3 -t 2>&1",
refreshonly => true,
subscribe => [ File["/etc/dhcp3/dhcpd.conf"], File["/etc/dhcp3"] ];
}

file {
"/etc/dhcp3/dhcpd.conf":
source => "puppet:///dist/dhcp/$hostname/dhcpd.conf";
"/etc/dhcp3":
source => "puppet:///dist/dhcp/include",
recurse => true;
}
}

Ideas?

-j

--
Jason Lavoie
Ratvarre sbe uver
ja...@oasys.net

[Peter Meier]

Hi

> I'm trying to have a exec dependency on a service object that would keep
> it from being restarted if the exec fails. Unfortunately, the service
> gets refreshed regardless whenever the exec is run, failure or not.
>
> I've tried various combinations of subscribe/require/notify, but can't
> find an elegant way to not restart the service when the dependency
> fails. I must be missing something obvious.
>
> We have a few needs for this model, here's a (simplified) example of one:
>

> [...]

> exec { "check-dhcpd-conf":
> command => "/usr/sbin/dhcpd3 -t 2>&1",
> refreshonly => true,
> subscribe => [ File["/etc/dhcp3/dhcpd.conf"], File["/etc/dhcp3"] ];
> }

> [...]
>
> Ideas?

don't set the command to refreshonly. this will trigger the exec only if
it get triggered by someone else otherwise the resource counts as
successful.

cheers pete

[Carl Caum]

Not to say "You're doing this all wrong!", but wouldn't this best be
handled in the init script? You can put it under puppet control.

[Jason Lavoie]

On 10/29, Carl Caum wrote:
>
> Not to say "You're doing this all wrong!", but wouldn't this best be
> handled in the init script? You can put it under puppet control.

I _knew_ someone would say that. :) In fact, for this particular
example I gave, the Debian init scripts already do the config check.

I do have more complex situations that I'd like to do this "in puppet"
rather than in the init script, mostly because of the rich dependency
declarations available in the puppet DSL.

I also avoided the init script option, as I was hoping to collect
the error output and propogate it (through tagmail.conf) to our ops
group. I'm not sure how/if this can be done for a service object.

[Jason Lavoie]

On 10/29, Peter Meier wrote:
> > I'm trying to have a exec dependency on a service object that would keep
> > it from being restarted if the exec fails. Unfortunately, the service
> > gets refreshed regardless whenever the exec is run, failure or not.

[...]

>
> don't set the command to refreshonly. this will trigger the exec only if
> it get triggered by someone else otherwise the resource counts as
> successful.

That doesn't seem to help. Here's a simpler test manifest:

service { dep:
hasstatus => true, status => "/bin/true status",
start => "/bin/true start", stop => "/bin/true stop",
subscribe => File["/tmp/dep"],
require => Exec["check-dep"],
}

exec { "check-dep":
command => "exit 1",
path => "/bin",
subscribe => File["/tmp/dep"],
}

file { "/tmp/dep":
content => "foo",
}

Shouldn't the "Skipping because of failed dependencies" in the output
below be absolute? Why is puppet still refreshing the Service[dep]
(in the very next log line)? Am I misunderstanding how the dependencies
should work?

-j

$ puppet --version
0.24.8
$ puppet --debug dep.pp
[...]
debug: //Service[dep]/require: requires Exec[check-dep]
debug: //Service[dep]/subscribe: subscribes to File[/tmp/dep]
debug: //Exec[check-dep]/subscribe: subscribes to File[/tmp/dep]
debug: //File[/tmp/dep]: File does not exist
debug: //File[/tmp/dep]/checksum: Initializing checksum hash
debug: //File[/tmp/dep]: Changing content
debug: //File[/tmp/dep]: 1 change(s)
debug: //File[/tmp/dep]: Creating checksum {md5}acbd18db4cc2f85cedef654fccc4a4d8
notice: //File[/tmp/dep]/content: defined 'content' as '{md5}acbd18db4cc2f85cedef654fccc4a4d8'
info: //File[/tmp/dep]: Scheduling refresh of Service[dep]
info: //File[/tmp/dep]: Scheduling refresh of Exec[check-dep]
debug: //Exec[check-dep]: Changing returns
debug: //Exec[check-dep]: 1 change(s)
debug: //Exec[check-dep]: Executing 'exit 1'
debug: Executing 'exit 1'
err: //Exec[check-dep]/returns: change from notrun to 0 failed: exit 1 returned 1 instead of 0 at /home/jlavoie/dep.pp:12
notice: //Exec[check-dep]: Triggering 'refresh' from 1 dependencies
debug: //Exec[check-dep]: Executing 'exit 1'
debug: Executing 'exit 1'
err: //Exec[check-dep]: Failed to call refresh on Exec[check-dep]: exit 1 returned 1 instead of 0 at /home/jlavoie/dep.pp:12
notice: //Service[dep]: Dependency exec[exit 1] has 1 failures
warning: //Service[dep]: Skipping because of failed dependencies
notice: //Service[dep]: Triggering 'refresh' from 1 dependencies
debug: Service[dep](provider=debian): Executing '/bin/true status'
debug: Service[dep](provider=debian): Executing '/bin/true stop'
debug: Service[dep](provider=debian): Executing '/bin/true start'
debug: Finishing transaction -606555348 with 2 changes

[Peter Meier]

Hi

> Shouldn't the "Skipping because of failed dependencies" in the output
> below be absolute? Why is puppet still refreshing the Service[dep]
> (in the very next log line)? Am I misunderstanding how the dependencies
> should work?

hmm, which version are you running?

> err: //Exec[check-dep]: Failed to call refresh on Exec[check-dep]: exit 1 returned 1 instead of 0 at /home/jlavoie/dep.pp:12
> notice: //Service[dep]: Dependency exec[exit 1] has 1 failures
> warning: //Service[dep]: Skipping because of failed dependencies
> notice: //Service[dep]: Triggering 'refresh' from 1 dependencies
> debug: Service[dep](provider=debian): Executing '/bin/true status'
> debug: Service[dep](provider=debian): Executing '/bin/true stop'
> debug: Service[dep](provider=debian): Executing '/bin/true start'
> debug: Finishing transaction -606555348 with 2 changes

I mean it says that is skipping the service beacause of failed
dependecies. strange. If it's with the latest puppet version I would
then file a bug, otherwise try first with the latest one.

cheers pete

[Jason Lavoie]

On 11/13, Peter Meier wrote:
> > Shouldn't the "Skipping because of failed dependencies" in the output
> > below be absolute? Why is puppet still refreshing the Service[dep]
> > (in the very next log line)? Am I misunderstanding how the dependencies
> > should work?
>
> hmm, which version are you running?

> > $ puppet --version
> > 0.24.8

The same thing happens with 0.25.1.

> I mean it says that is skipping the service beacause of failed
> dependecies. strange. If it's with the latest puppet version I would
> then file a bug, otherwise try first with the latest one.

http://projects.reductivelabs.com/issues/2814

Thanks for the sanity check!

-j

[Luke Kanies]

The short answer (as I put on the ticket) is that this is behaving as
designed, because otherwise you're running in an inconsistent state.

The easy fix here is to not have the service depend on the file, but
rather only on the exec - the exec subscribes to the file, the service
subscribes to the exec. If the exec fails, nothing happens, if it
succeeds you're all good.

--
Life isn't fair. It's just fairer than death, that's all.
-- William Goldman
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

[Jason Lavoie]

On 11/13, Luke Kanies wrote:
> On Nov 12, 2009, at 11:58 AM, Jason Lavoie wrote:
> > Shouldn't the "Skipping because of failed dependencies" in the output
> > below be absolute? Why is puppet still refreshing the Service[dep]
> > (in the very next log line)? Am I misunderstanding how the
> > dependencies
> > should work?
>
> The short answer (as I put on the ticket) is that this is behaving as
> designed, because otherwise you're running in an inconsistent state.

That makes sense. Thanks for the definitive answer.

> The easy fix here is to not have the service depend on the file, but
> rather only on the exec - the exec subscribes to the file, the service
> subscribes to the exec. If the exec fails, nothing happens, if it
> succeeds you're all good.

Yes, but the exec runs every time. If I then add refreshonly=>true to
the exec, puppet still refreshes the service when the exec fails.

-j