puppet agent fails to self-restart on config change

[SCHAER Frederic]

Hi,

 

For some time now, we’ve seen this kind of logs in our machines :

(note: we do manage the puppet agent config with puppet)

 

Mar 29 13:25:50 gate01 puppet-agent[2684]: Processing restart

Mar 29 13:25:50 gate01 puppet-agent[2684]: Puppet::Agent::Locker.running? is deprecated as it is inherently unsafe. The only safe way to know if the lock is locked is to try lock and perform some action and then handle the LockError that may result.

Mar 29 13:25:50 gate01 puppet-agent[2684]:    (location: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:37:in `running?')

Mar 29 13:25:51 gate01 puppet-agent[38303]: Applied catalog in 16.78 seconds

Mar 29 13:26:00 gate01 puppet-agent[2684]: Config file /etc/puppetlabs/puppet/puppet.conf changed; triggering re-parse of all config files.

Mar 29 13:44:16 gate01 puppet-agent[2684]: Shutdown/restart in progress (:restart_requested); skipping run

 

Than puppet never runs again because of this stuck restart until we manually restart the agent.

puppet-agent-5.5.12 here.

 

This happens on both el6 and el7 hosts.

 

Bad thing is that this even happens when puppet is upgraded during our install process … and we get freshly installed nodes with stuck puppets…

Off course, this did not happen before.

 

Would anyone know of a way to get rid of this damn “can’t restart” loop ?

 

Regards

 

Frédéric Schaer

CEA/Saclay

DRF/IRFU/DEDIP/LIS

 

[Aleksandr Smirnov]

Same issue:

May 21 11:26:51 vm01 puppet-agent[419]: Shutdown/restart in progress (:restart_requested); skipping run

May 21 12:26:51 vm01 puppet-agent[419]: Shutdown/restart in progress (:restart_requested); skipping run

May 21 13:26:51 vm01 puppet-agent[419]: Shutdown/restart in progress (:restart_requested); skipping run

[Yvan Broccard]

I experience the same issue here, sometimes agents are stuck in this Shutdown/restart loop, especially if puppet agent changed a config value in puppet.conf.

We're using Puppet 6 now here and this issue is new with Puppet 6. In the past, with puppet serie 4, we never experienced this.

I've updated the puppet-agent within the 6.10.x serie, hoping that a fix would be delivered but not so far.

Our clients nodes are mainly on CentOS 7.6/7.7

I'm surprised by the few similar issue I get by browsing the Internet for this issue.

Cheers

Yvan

[Josh Cooper]

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/8fcbe059-90ce-4afa-9806-7a2d837eb3ac%40googlegroups.com.

I was able to reproduce this a few months ago, but didn't find the cause. Could someone file a JIRA ticket if there isn't one already?

Josh

--

Josh Cooper | Software Engineer

jo...@puppet.com | @coopjn