Jira (PUP-11000) Race condition with agent_disabled_lockfile

16 views
Skip to first unread message

George Campbell (Jira)

unread,
Mar 31, 2021, 7:28:05 AM3/31/21
to puppe...@googlegroups.com
George Campbell created an issue
 
Puppet / Bug PUP-11000
Race condition with agent_disabled_lockfile
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2021/03/31 4:27 AM
Priority: High High
Reporter: George Campbell

Puppet Version: Affects all versions
Puppet Server Version:
OS Name/Version: CentOS 7

This is essentially the same issue as seen in a an old PR which was lost over the years -https://github.com/puppetlabs/puppet/pull/1158 where rapid or parallel puppet runs can cause an already existing agent_disabled_lockfile to get truncated 

Desired Behavior:

I do not believe puppet actually needs to be creating a resource for this file, the agent_catalog_run_lockfile has a setting type of string for this reason (https://github.com/puppetlabs/puppet/blob/main/lib/puppet/defaults.rb#L1669) I'd like to apply the same setting type to agent_disabled_lockfile

Actual Behavior:

Examples:
puppet agent --test --trace --debug --logdest /root/puppet-debug.log

Lockfile being created (grep agent_disabled puppet-debug.log):

Note: File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/ensure (debug): created

2021-03-31 10:49:33 +0000 Puppet (debug): Using settings: adding file resource 'agent_disabled_lockfile': 'File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]{:path=>"/opt/puppetlabs/puppet/cache/state/agent_disabled.lock", :ensure=>:file, :loglevel=>:debug, :links=>:follow, :backup=>false}'
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/seluser (debug): Found seluser default 'system_u' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/selrole (debug): Found selrole default 'object_r' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/seltype (debug): Found seltype default 'usr_t' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/selrange (debug): Found selrange default 's0' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock] (debug): Adding autorequire relationship with File[/opt/puppetlabs/puppet/cache/state]
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/ensure (debug): created

 Strace from guilty puppet agent --test

314978 02:46:02.236891 open("/opt/puppetlabs/puppet/cache/state/agent_disabled.lock", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 18</opt/puppetlabs/puppet/cache/state/agent_disabled.lock> <0.000071>

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

George Campbell (Jira)

unread,
Mar 31, 2021, 7:41:03 AM3/31/21
to puppe...@googlegroups.com
George Campbell updated an issue
Change By: George Campbell
*Puppet Version: Affects all versions*
*Puppet Server Version:*
*OS Name/Version: CentOS 7*

This is essentially the same issue as seen in a an old PR which was lost over the years -[https://github.com/puppetlabs/puppet/pull/1158] where rapid or parallel puppet runs can cause an already existing {{agent_disabled_lockfile}} to get truncated 

*Desired Behavior:*

I do not believe puppet actually needs to be creating a resource for this file, the {{agent_catalog_run_lockfile}} has a setting type of {{string}} for this reason ([https://github.com/puppetlabs/puppet/blob/main/lib/puppet/defaults.rb#L1669]) I'd like to apply the same setting type to {{agent_disabled_lockfile}}

*Actual Behavior:*


Examples:
{{puppet agent --test --trace --debug --logdest /root/puppet-debug.log}}

Lockfile being created (grep agent_disabled puppet-debug.log):

Note: {{File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/ensure (debug): created}}
{code:java}2021-03-31 10:49:33 +0000 Puppet (debug): Using settings: adding file resource 'agent_disabled_lockfile': 'File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]{:path=>"/opt/puppetlabs/puppet/cache/state/agent_disabled.lock", :ensure=>:file, :loglevel=>:debug, :links=>:follow, :backup=>false}'

2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/seluser (debug): Found seluser default 'system_u' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/selrole (debug): Found selrole default 'object_r' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/seltype (debug): Found seltype default 'usr_t' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/selrange (debug): Found selrange default 's0' for /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock] (debug): Adding autorequire relationship with File[/opt/puppetlabs/puppet/cache/state]
2021-03-31 10:49:34 +0000 /File[/opt/puppetlabs/puppet/cache/state/agent_disabled.lock]/ensure (debug): created
{code}

 Strace from guilty {{puppet agent --test}}
{code:java} 314978 02 428070 10 : 46 49 : 02 34 . 236891 313537 open("/opt/puppetlabs/puppet/cache/state/agent_disabled.lock", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 18</opt/puppetlabs/puppet/cache/state/agent_disabled.lock> <0. 000071 000076 >
{code}

George Campbell (Jira)

unread,
Mar 31, 2021, 7:41:03 AM3/31/21
to puppe...@googlegroups.com
[https://github.com/puppetlabs/puppet/pull/1158] where rapid or parallel puppet runs can cause an already existing {{agent_disabled_lockfile}} to get truncated 
{code:java}428070 10:49:34.313537 open("/opt/puppetlabs/puppet/cache/state/agent_disabled.lock", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 18</opt/puppetlabs/puppet/cache/state/agent_disabled.lock> <0.000076>
{code}

Josh Cooper (Jira)

unread,
Apr 1, 2021, 1:37:03 PM4/1/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 7.6.0
Fix Version/s: PUP 6.22.0

Josh Cooper (Jira)

unread,
Apr 1, 2021, 1:42:03 PM4/1/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Release Notes: Bug Fix
Release Notes Summary: Fixes a race condition that could cause an agent to become disabled and no longer enforce desired state.

Josh Cooper (Jira)

unread,
Apr 2, 2021, 6:27:04 PM4/2/21
to puppe...@googlegroups.com
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Josh Cooper (Jira)

unread,
Apr 2, 2021, 6:28:02 PM4/2/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Release Notes Summary:
Fixes a race condition that could cause an agent to become disabled and no longer enforce desired state.


This was a community contribution from https://github.com/gcampbell12

Claire Cadman (Jira)

unread,
Apr 12, 2021, 9:24:01 AM4/12/21
to puppe...@googlegroups.com
Claire Cadman updated an issue
Change By: Claire Cadman
Labels: doc_reviewed

Josh Cooper (Jira)

unread,
Apr 14, 2021, 12:52:02 PM4/14/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Community Contributors: gcampbell12
Reply all
Reply to author
Forward
0 new messages