Agents seem to hang during updates, Run already in progress; skipping

[Vorik]

Hi,

Quite a few agents on my servers seem to hang during updates.

They show up as unresponsive in Dashboard and when I try to run
puppet agent --test
, I get this output:

--------%<--------
# puppet agent --test
notice: Ignoring --listen on onetime run
notice: Run of Puppet configuration client already in progress;
skipping


In the logs, I can see that there were no runs for a few days:
--------%<--------
Oct 25 16:14:41 tom02 puppet-agent[15084]: Finished catalog run in
2.63 seconds
Oct 25 16:44:53 tom02 puppet-agent[15084]: Finished catalog run in
2.16 seconds
Oct 25 17:15:07 tom02 puppet-agent[15084]: Finished catalog run in
2.23 seconds
Oct 25 17:45:21 tom02 puppet-agent[15084]: Finished catalog run in
2.71 seconds
Oct 27 07:43:21 tom02 puppet-agent[24326]: Ignoring --listen on
onetime run
Oct 27 07:43:21 tom02 puppet-agent[24326]: Run of Puppet configuration
client already in progress; skipping

Background data:
--------%<--------
Server:
Red Hat Enterprise Linux Server release 6.1 (Santiago)
# rpm -qa | grep puppet
puppet-2.7.5-1.el6.noarch
puppet-dashboard-1.2.1-1.el6.noarch
puppet-server-2.7.5-1.el6.noarch

Client:
Red Hat Enterprise Linux Server release 6.1 (Santiago)
# rpm -qa | grep puppet
puppet-2.7.5-1.el6.noarch

This happens on several servers, but not all.

Any help is very much appreciated!

Thanks,
Ger Apeldoorn

[Radek]

Hi Vorik,

Did you solve your problem? I'm having the same issue with the 2.7.9 on CentOS and Amazon Linux.

Puppet agent stops polling for configs and it tells me it's in progress when I run it manually.

--
Radek

[Andreas N]

I think I have the same problem onUbuntu 10.04.3 and 2.7.9, as I just posted to this group. The symptoms seem identical and in ps -ef I see that the puppet agent spawned an sh process that became defunct. This seems to be the cause for the hanging agent, but what causes that zombie process I have no idea.

[Ger Apeldoorn]

Hi Radek,

It's a bug that acts up on a specific kernel; see: http://projects.puppetlabs.com/issues/10819

I've upgraded to Puppet Enterprise and do not have this issue anymore.

Regards,
Ger.

[Aaron Grewell]

That's interesting - PE isn't affected?

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/GXaYE-tuyMUJ.
>
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.

[Michael Stahnke]

On Thu, Jan 5, 2012 at 9:12 AM, Aaron Grewell <aaron....@gmail.com> wrote:
> That's interesting - PE isn't affected?
>
> On Thu, Jan 5, 2012 at 12:26 AM, Ger Apeldoorn <in...@gerapeldoorn.nl> wrote:
>> Hi Radek,
>>
>> It's a bug that acts up on a specific kernel; see:
>> http://projects.puppetlabs.com/issues/10819
>>
>> I've upgraded to Puppet Enterprise and do not have this issue anymore.
>>

If that's true, that's probably mostly coincidence. The puppet in PE
is the same puppet that's available on downloads.puppetlabs.com. It's
not that I'm trying to say don't use PE, please do. But there is no
magic "make this work muhahahahaha" patch included.

[Michael Stahnke]

It would be helpful to note what versions are involved.

PE 2.0 ships with Puppet 2.7.6.

Puppet 2.7.9 you said you had an issue. I'd like to find out more.

Another option to work-around an issue like this might be to run
puppet from cron rather than as a daemon.

[Ger Apeldoorn]

In my 1st post, the versions of the problematic configurations are mentioned.

During the PE upgrade, I also created new configfiles without 'listen = true'. This is what probably fixed it for me, 'listen' is specifically mentioned in the bugreport.

Ger.

[Jo Rhett]

You can get the hang without using listen.  Any time ruby has two file handles open, the way it handles /proc changes -- so you can hit the bug in #10418 without enabling listen. I was able to do so quite easily. It's not a consistent workaround.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Aczw695ilTIJ.

To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

-- 

Jo Rhett

Net Consonance : consonant endings by net philanthropy, open source and other randomness

[MF]

Hi,

Has there been any updates for this problem? I have a new installation of Puppet 2.7 on RHEL 6.1 and 6.2 and I am experiencing the same problem with listen=true using Dashboard and puppet kick. The kernels that I have installed are: 2.6.32-220.el6.x86_64, 2.6.32-220.7.1.el6.x86_64.

Thanks