Jira (PUP-10367) Recursive copy of directory fails in file resource

29 views
Skip to first unread message

Bert Hajee (Jira)

unread,
Mar 12, 2020, 5:17:03 AM3/12/20
to puppe...@googlegroups.com
Bert Hajee created an issue
 
Puppet / Bug PUP-10367
Recursive copy of directory fails in file resource
Issue Type: Bug Bug
Affects Versions: PUP 6.14.0
Assignee: Unassigned
Components: Types and Providers
Created: 2020/03/12 2:16 AM
Priority: Critical Critical
Reporter: Bert Hajee

Puppet Version: 6.14.0
Puppet Server Version: NA
OS Name/Version: Centos 7

In Puppet 6.14 copying a puppet served directory fails. Here is an example

{{ file { '/a':}}
  ensure  => 'directory',
  source  => "puppet:///modules/test",
  recurse => true,
  purge   => true,
{{ }}}

 

Now fails with this error

[root@rcudb vagrant]# puppet apply test_case.pp
Notice: Compiled catalog for rcudb.example.com in environment production in 0.05 seconds
Notice: /Stage[main]/Main/File[/a]/ensure: created
Error: Could not set 'file' on ensure: Is a directory @ io_fread - /etc/puppetlabs/code/environments/production/modules/test/files
Error: Could not set 'file' on ensure: Is a directory @ io_fread - /etc/puppetlabs/code/environments/production/modules/test/files
Wrapped exception:
Is a directory @ io_fread - /etc/puppetlabs/code/environments/production/modules/test/files
Error: /Stage[main]/Main/File[/a/a.a]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: Is a directory @ io_fread - /etc/puppetlabs/code/environments/production/modules/test/files
Notice: Applied catalog in 0.42 seconds

 

Desired Behavior:

With puppet version puppet-agent.x86_64 0:6.13.0-1.el7 This works as expected:

[root@rcudb vagrant]# puppet apply test_case.pp
Notice: Compiled catalog for rcudb.example.com in environment production in 0.06 seconds
Notice: /Stage[main]/Main/File[/a]/seltype: seltype changed 'etc_runtime_t' to 'default_t'
Notice: /Stage[main]/Main/File[/a/a.a]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: Applied catalog in 0.47 seconds

 

 
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Bert Hajee (Jira)

unread,
Mar 12, 2020, 5:54:03 AM3/12/20
to puppe...@googlegroups.com
Bert Hajee updated an issue
Change By: Bert Hajee
*Puppet Version: 6.14.0*
*Puppet Server Version: NA*
*OS Name/Version: Centos 7*


In Puppet 6.14 copying a puppet served directory fails. Here is an example

{{   file { '/a': }}
{{  ensure  => 'directory',}}
{{  source  => "puppet:///modules/test",}}
{{  recurse => true,}}
{{  purge   => true,}}
 {{ } }}

 
Now fails with this error

{{[root@rcudb vagrant]# puppet apply test_case.pp}}
{{Notice: Compiled catalog for rcudb.example.com in environment production in 0.05 seconds}}
{{Notice: /Stage[main]/Main/File[/a]/ensure: created}}
{{{color:#de350b}Error: Could not set 'file' on ensure: Is a directory @ io_fread - /etc/puppetlabs/code/environments/production/modules/test/files{color}}}
{{{color:#de350b}Error: Could not set 'file' on ensure: Is a directory @ io_fread - /etc/puppetlabs/code/environments/production/modules/test/files{color}}}
{{{color:#de350b}Wrapped exception:{color}}}
{{{color:#de350b}Is a directory @ io_fread - /etc/puppetlabs/code/environments/production/modules/test/files{color}}}
{{{color:#de350b}Error: /Stage[main]/Main/File[/a/a.a]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: Is a directory @ io_fread -{color} /etc/puppetlabs/code/environments/production/modules/test/files}}

{{Notice: Applied catalog in 0.42 seconds}}

 

*Desired Behavior:*


With puppet version puppet-agent.x86_64 0:6.13.0-1.el7 This works as expected:

[root@rcudb vagrant]# puppet apply test_case.pp
Notice: Compiled catalog for rcudb.example.com in environment production in 0.06 seconds
Notice: /Stage[main]/Main/File[/a]/seltype: seltype changed 'etc_runtime_t' to 'default_t'
Notice: /Stage[main]/Main/File[/a/a.a]/ensure: defined content as '\{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: Applied catalog in 0.47 seconds

 

 

Bert Hajee (Jira)

unread,
Mar 12, 2020, 5:55:04 AM3/12/20
to puppe...@googlegroups.com

Bert Hajee (Jira)

unread,
Mar 12, 2020, 5:56:03 AM3/12/20
to puppe...@googlegroups.com

Bert Hajee (Jira)

unread,
Mar 12, 2020, 5:56:03 AM3/12/20
to puppe...@googlegroups.com

Sean Ford (Jira)

unread,
Mar 12, 2020, 8:53:03 AM3/12/20
to puppe...@googlegroups.com
Sean Ford commented on Bug PUP-10367
 
Re: Recursive copy of directory fails in file resource

We are running into this same problem on ubuntu 16.04.

While we use Puppet agent on our hosts, Beaker uses puppet apply for acceptance testing of Puppet modules. So, all our tests are failing due to this after upgrading to Puppet 6.14.0.

Our workaround is downgrading to Puppet 6.13.0 just for our Beaker testing.

Bert Hajee (Jira)

unread,
Mar 12, 2020, 11:22:02 AM3/12/20
to puppe...@googlegroups.com
Bert Hajee commented on Bug PUP-10367

So it does not occur when using the puppet agent?

Sean Ford (Jira)

unread,
Mar 12, 2020, 11:37:03 AM3/12/20
to puppe...@googlegroups.com
Sean Ford commented on Bug PUP-10367

So it does not occur when using the puppet agent?

Hard to say for certain... but... we haven't seen this problem on any host that is using puppet agent -> server. The only area I have seen this problem is with puppet apply.

Puppet 6.14.0 did introduce changes around puppet apply and streaming file content:
https://tickets.puppetlabs.com/browse/PUP-8338

Maybe this is related?

Sean Ford (Jira)

unread,
Mar 12, 2020, 11:38:03 AM3/12/20
to puppe...@googlegroups.com
Sean Ford commented on Bug PUP-10367

Looks like Josh Cooper did PUP-8338, and I see him watching this ticket. So maybe this is on their radar.

Josh Cooper (Jira)

unread,
Mar 12, 2020, 1:33:03 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
 
Change By: Josh Cooper
Team: Coremunity

Josh Cooper (Jira)

unread,
Mar 12, 2020, 1:33:03 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10367
 
Re: Recursive copy of directory fails in file resource

In the puppet apply case, we try to retrieve file content using metadata.path, but in the recusive file case path is the initial point of recursion, it's more like base_path. We need to call metadata.full_path instead, which joins the path and relative_path.

[{"path"=>
 
   "/etc/puppetlabs/code/environments/production/modules/filetest/files/dir",
 
  "relative_path"=>".",
 
  "links"=>"manage",
 
  "owner"=>996,
 
  "group"=>994,
 
  "mode"=>420,
 
  "checksum"=>{"type"=>"ctime", "value"=>"{ctime}2020-03-12 17:06:06 UTC"},
 
  "type"=>"directory",
 
  "destination"=>nil},
 
 {"path"=>
 
   "/etc/puppetlabs/code/environments/production/modules/filetest/files/dir",
 
  "relative_path"=>"dir",
 
  "links"=>"manage",
 
  "owner"=>996,
 
  "group"=>994,
 
  "mode"=>420,
 
  "checksum"=>{"type"=>"ctime", "value"=>"{ctime}2020-03-12 17:06:06 UTC"},
 
  "type"=>"directory",
 
  "destination"=>nil},
 
 {"path"=>
 
   "/etc/puppetlabs/code/environments/production/modules/filetest/files/dir",
 
  "relative_path"=>"dir/file",
 
  "links"=>"manage",
 
  "owner"=>996,
 
  "group"=>994,
 
  "mode"=>420,
 
  "checksum"=>
 
   {"type"=>"md5", "value"=>"{md5}d41d8cd98f00b204e9800998ecf8427e"},
 
  "type"=>"file",
 
  "destination"=>nil}]

Josh Cooper (Jira)

unread,
Mar 12, 2020, 1:33:04 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.15.0

Josh Cooper (Jira)

unread,
Mar 12, 2020, 1:39:02 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Platform Core KANBAN

Josh Cooper (Jira)

unread,
Mar 12, 2020, 1:39:03 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper assigned an issue to Josh Cooper
Change By: Josh Cooper
Assignee: Josh Cooper

Josh Cooper (Jira)

unread,
Mar 12, 2020, 1:43:04 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10367
 
Re: Recursive copy of directory fails in file resource 

diff --git a/lib/puppet/type/file/source.rb b/lib/puppet/type/file/source.rb
 
index bb215d59c4..55db103c6b 100644
 
--- a/lib/puppet/type/file/source.rb
 
+++ b/lib/puppet/type/file/source.rb
 
@@ -257,7 +257,7 @@ module Puppet
 
 
 
     def each_chunk_from(&block)
 
       if Puppet[:default_file_terminus] == :file_server && scheme == 'puppet' && (uri.host.nil? || uri.host.empty?)
 
-        chunk_file_from_disk(metadata.path, &block)
 
+        chunk_file_from_disk(metadata.full_path, &block)
 
       elsif local?
 
         chunk_file_from_disk(full_path, &block)
 
       else

Josh Cooper (Jira)

unread,
Mar 12, 2020, 4:21:04 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Epic Link: PUP-8550

Josh Cooper (Jira)

unread,
Mar 12, 2020, 6:09:03 PM3/12/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Release Notes: Bug Fix
Release Notes Summary: Fixes a regression in puppet 6.14.0 that prevented the "puppet apply" application from recursively copying a directory from a module using a "puppet://" URL.

Simon Guest (Jira)

unread,
Mar 18, 2020, 11:41:05 PM3/18/20
to puppe...@googlegroups.com
Simon Guest commented on Bug PUP-10367
 
Re: Recursive copy of directory fails in file resource

Using puppet apply here, hitting the same problem, which seems to be a showstopper for use of puppet 6.14 in a distributed (agentless) scenario.  We're reverting to 6.13 for now, which is working fine.

Melissa Stone (Jira)

unread,
Mar 19, 2020, 1:03:04 PM3/19/20
to puppe...@googlegroups.com

merged into master at f2740a0f456e0d46c05e78cf81e8a925495dbf0d

Melissa Stone (Jira)

unread,
Mar 20, 2020, 7:12:04 PM3/20/20
to puppe...@googlegroups.com

passed ci as a part of puppet-agent 6.14.0.94.g47a3ec5d
Reply all
Reply to author
Forward
0 new messages