Re: Apparently service not restarted although is supposed to

[jcbollinger]

On Monday, June 18, 2012 4:53:23 AM UTC-5, Javi wrote:

Scenario: install SNMP (agent and daemon), change config file and restart of service.

What's wrong? The logs showed that AFTER the new config file is applied, the service (snmpd) is restarted, but looks like this is not true.

The log you presented shows that Puppet took the action it thought appropriate for refreshing the service, and that action returned a success code.  If that didn't actually (re)start the service then probably your declarations for it are inconsistent with the actual service implementation (most likely with the initscript), or else something else happening on the node is interfering.

Jun 15 17:08:21 test-3 puppet-agent[1219]: (/Stage[main]/Apt::Service/Exec[/usr/bin/apt-get update]/returns) executed successfully
Jun 15 17:08:44 test-3 puppet-agent[1219]: (/Stage[main]/Snmp::Install/Package[snmp]/ensure) ensure changed 'purged' to 'present'

[... many complaints from snmpd ...]

Why is snmpd logging anything at that point?  Does installing the package automatically start the service?  That would be horrible!

Jun 15 17:09:01 test-3 puppet-agent[1219]: (/Stage[main]/Snmp::Install/Package[snmpd]/ensure) ensure changed 'purged' to 'present'
Jun 15 17:09:01 test-3 puppet-agent[1219]: FileBucket adding /etc/snmp/snmpd.conf as {md5}a5007383dd9c4ef73500e3df8c080665
Jun 15 17:09:01 test-3 puppet-agent[1219]: (/Stage[main]/Snmp::Config/File[/etc/snmp/snmpd.conf]) Filebucketed /etc/snmp/snmpd.conf to puppet with sum a5007383dd9c4ef73500e3df8c080665
Jun 15 17:09:02 test-3 puppet-agent[1219]: (/Stage[main]/Snmp::Config/File[/etc/snmp/snmpd.conf]/content) content changed '{md5}a5007383dd9c4ef73500e3df8c080665' to '{md5}6a797811e82b5f411af1093ea6336a04'
Jun 15 17:09:02 test-3 puppet-agent[1219]: (/Stage[main]/Snmp::Config/File[/etc/snmp/snmpd.conf]) Scheduling refresh of Service[snmpd]
Jun 15 17:09:02 test-3 puppet-agent[1219]: (/Stage[main]/Snmp::Service/Service[snmpd]/ensure) ensure changed 'stopped' to 'running'

The "refresh" Puppet performed was an attempt to change the service from the "stopped" state to the "running" state.  For Debian-family distros, that probably means running '/etc/init.d/snmpd start'.  Note: not restart, because Puppet doesn't think the service is running yet.  That's suspicious, because snmpd was just observed logging all sorts of complaints about its config file.
 

Jun 15 17:09:02 test-3 puppet-agent[1219]: (/Stage[main]/Snmp::Service/Service[snmpd]) Triggered 'refresh' from 1 events
Jun 15 17:09:02 test-3 puppet-agent[1219]: Finished catalog run in 73.61 seconds

So the order of operations appears correct, but the refresh of Service['snmpd'] doesn't seem to take.  I see a couple of plausible explanations:

• The snmpd service is already running when Puppet tries to refresh it, but Puppet thinks it's stopped.  Puppet therefore issues the 'start' command to the initscript, which typically does nothing if the service is already running.  I'd put my bet here.
  
• The initscript 'start' command returns an exit code indicating success, even though it failed to start the service for some unknown reason.
  

Let's focus now on the service declaration:

puppet-1.dev.jj.com:/etc/puppet/modules/snmp/manifests/service.pp

class snmp::service {
        service { $snmp::params::snmp_service_name:
                ensure => running,
                hasstatus => true,
                hasrestart => true,

Those 'hasstatus' and 'hasrestart' declarations tell Puppet that the service's init script supports the 'restart' and 'status' commands, and that they return codes at least approximately consistent with those specified by the LSB.  In particular, Puppet pays attention to whether the those commands' exit codes are zero or nonzero.  You should verify that the 'status' command returns 0 if and only if the service is already running, and that the 'restart' command returns 0 if and only if the service is successfully (re)started.
 

        # Boot time
                enable => true,
                require => Class["snmp::config"],

That require is mostly redundant, but harmless.
 

        }
}

Overall, the manifests look good, subject to the caveats I already discussed.

If the problem is that installing the package starts snmpd behind Puppet's back, then probably the situation will resolve itself on the second Puppet run.  Alternatively, perhaps there is a way to prevent package install from starting the service.

If the problem is that the init script's 'status' or 'restart' command is not working properly, then you can either fix the script between installing the package and managing the service, or you can instead specify an alternative start and/or status command via the Service's 'status' and 'restart' parameters.


John

[Javi Legido]

Amazing!

Many thanks, I will check all your suggestions, but I use similar
modules for other services (such as puppet, ssh, ntp...) On debian
nodes (all of them are clones) smoothly, but SNMP init script can be
buggy.

Thanks again

> that probably means running '/etc/init.d/snmpd start'. Note: not *re*start,

>
> because Puppet doesn't think the service is running yet. That's
> suspicious, because snmpd was just observed logging all sorts of complaints
>
> about its config file.
>
>
>> Jun 15 17:09:02 test-3 puppet-agent[1219]:
>> (/Stage[main]/Snmp::Service/Service[snmpd]) Triggered 'refresh' from 1
>> events
>> Jun 15 17:09:02 test-3 puppet-agent[1219]: Finished catalog run in 73.61
>> seconds
>>
>
> So the order of operations appears correct, but the refresh of
> Service['snmpd'] doesn't seem to take. I see a couple of plausible
> explanations:
>

> - The snmpd service is already running when Puppet tries to refresh it,

> but Puppet thinks it's stopped. Puppet therefore issues the 'start'
> command to the initscript, which typically does nothing if the service is
>
> already running. I'd put my bet here.

> - The initscript 'start' command returns an exit code indicating

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/oN9V3JEezzAJ.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>

--
Sent from my mobile device

[Javi Legido]

A dirty debian based workaround just in case somebody is experiencing
same issue:

class snmp::service {
exec { "SNMP custom restart":
command => "/etc/init.d/snmpd restart",
subscribe => Class["snmp::config"],
refreshonly => true
}
}

Best regards



2012/6/19 Javi Legido <ja...@legido.com>: