Jira (PUP-2836) Incorrect lockfile placemen

2 views
Skip to first unread message

Nick Moriarty (JIRA)

unread,
Feb 19, 2015, 4:44:42 AM2/19/15
to puppe...@googlegroups.com
Nick Moriarty commented on Bug PUP-2836
 
Re: Incorrect lockfile placemen

We're hitting this issue occasionally; at present, we're planning to work around it by changing agent_catalog_run_lockfile to $rundir/agent_catalog_run.lock.

At present, the default value of this is $statedir/agent_catalog_run.lock, which persists across reboots - I'd recommend this change to $rundir/agent_catalog_run.lock (which is where the main pidfiles live) so it doesn't persist.

This issue crops up occasionally (we're seeing it in 3.7.3) - even with the sanity checking of lockfiles, it's possible to have a catalog run lockfile that persists across a reboot, containing a valid PID which references a completely different process after the reboot. In this case, Puppet may never run (consider the case where it ends up pointing at sshd, for example).

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a)
Atlassian logo

Nick Moriarty (JIRA)

unread,
Feb 19, 2015, 4:45:38 AM2/19/15
to puppe...@googlegroups.com
Nick Moriarty updated an issue
 
Puppet / Bug PUP-2836
Incorrect lockfile placement
Change By: Nick Moriarty
Affects Version/s: PUP 3.7.3
Summary: Incorrect lockfile  placemen  placement

Ioannis (JIRA)

unread,
Nov 12, 2015, 12:25:16 PM11/12/15
to puppe...@googlegroups.com
Ioannis commented on Bug PUP-2836
 
Re: Incorrect lockfile placement

Is this a lock file or a pid file? It seems to be purposed for a lock but also contains a PID. As suggested above this file should move under /var/run or /var/lock which would at least allow puppet to start clean on reboot. May go a step further and do an exclusive flock, so puppet can be (re)started even without a reboot. The latter isn't supported on all platforms but why not use it when it's available?

This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)
Atlassian logo

Nick Moriarty (JIRA)

unread,
Nov 13, 2015, 4:21:05 AM11/13/15
to puppe...@googlegroups.com
Nick Moriarty commented on Bug PUP-2836
 
Re: Incorrect lockfile placement

It's a lockfile. It signals that configuration is currently being applied, and so another run shouldn't be started. I believe it only contains a PID to allow Puppet to attempt to check whether it's actually valid - it checks that there is actually a process with that PID, in case the agent crashed and left the lockfile behind.

Unfortunately, I've found in the past (this may or may not still be true) that Puppet doesn't seem to validate the process name, so you can end up in a situation where (especially after a reboot if the lockfile persists) another process ends up using that PID, which causes Puppet never to run again.

Using $rundir (which should be defined somewhere that doesn't persist across reboots), as mentioned above, in the path to this lockfile has cured the problem in our case; that should only require changing the default.

Russell Mull (JIRA)

unread,
May 16, 2017, 2:44:04 PM5/16/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Labels: triaged
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Russell Mull (JIRA)

unread,
May 16, 2017, 2:45:03 PM5/16/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Team: Agent

Moses Mendoza (JIRA)

unread,
May 18, 2017, 1:49:06 PM5/18/17
to puppe...@googlegroups.com
Moses Mendoza updated an issue
Change By: Moses Mendoza
Labels: triaged

Josh Cooper (JIRA)

unread,
Mar 16, 2018, 2:38:03 PM3/16/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sub-team: Coremunity
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Josh Cooper (JIRA)

unread,
Mar 19, 2018, 12:19:03 PM3/19/18
to puppe...@googlegroups.com
Josh Cooper assigned an issue to Unassigned
Change By: Josh Cooper
Assignee: Kylo Ginsberg

Josh Cooper (JIRA)

unread,
Jan 22, 2020, 11:50:04 PM1/22/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-2836
 
Re: Incorrect lockfile placement

Puppet now correctly reclaims a stale lockfile (see PUP-9247 and PUP-9691). It is unlikely that we'll change the location of the lockfile as that is a way that external programs (choria, etc) disable background agent runs. I'm going the close this.

Reply all
Reply to author
Forward
0 new messages