Jira (PUP-8338) Puppet apply doesn't stream file content

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-8338 Puppet apply doesn't stream file content Change By: Josh Cooper Summary: Puppet apply doesn't stream  remote  file content Add Comment

This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-8338 Puppet apply doesn't stream file content Change By: Josh Cooper

*Puppet Version:* Puppet 4 (at least) and up *Puppet Server Version:* n/a *OS Name/Version:* all When running {{puppet apply}} with a file resource  whose  and  source parameter  is {{http(s)}} or {{puppet}} , then puppet will load the entire file into memory, instead of streaming it.  Note the {{puppet}} case only happens if a host isn't specified, as that is interpreted to mean "use the local fileserver". This was noticed while researching why the bug described in PUP-7482 only affected {{puppet agent}} but not {{puppet apply}}. *Desired Behavior:* When managing files with a {{source}} parameter, puppet apply should always stream file content from all of the following sources {{<absolute path>}} local file {{file://<URI-encoded path>}} local file {{puppet:///modules/mymodule}} local file {{http?://host:port/path/to/file}} remote file *Actual Behavior:* {{puppet apply}} does not stream file content from http-based sources. See https://github.com/puppetlabs/puppet/blob/5.3.3/lib/puppet/type/file/source.rb#L274-L276. At first glance it appears the {{Puppet::Type::File::ParameterSource#content}} method can be deleted entirely, and should always rely on {{ chunk_file_from_source chunk_file_from_disk }}  which will invoke the  or  {{ terminus_selector.rb chunk_file_from_source }}  to determine the correct source .

Add Comment

[Rob Braden (JIRA)]

Rob Braden updated an issue

Puppet / PUP-8338 Puppet apply doesn't stream file content

Change By: Rob Braden Sprint: Coremunity Hopper

Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-8338 Puppet apply doesn't stream file content

Change By: Josh Cooper Fix Version/s: PUP 6.14.0

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper assigned an issue to Josh Cooper

Puppet / PUP-8338 Puppet apply doesn't stream file content

Change By: Josh Cooper Assignee: Josh Cooper

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-8338 Puppet apply doesn't stream file content

Change By: Josh Cooper Sprint: Coremunity Hopper Platform Core KANBAN

Add Comment

[Melissa Stone (JIRA)]

Melissa Stone commented on PUP-8338

Re: Puppet apply doesn't stream file content Merged into puppet master at https://github.com/puppetlabs/puppet/commit/42fff230e38d5d23fa6b2b25668de536219c7eba

Add Comment

[Melissa Stone (JIRA)]

Melissa Stone commented on PUP-8338

Re: Puppet apply doesn't stream file content

This has passed ci as a part of puppet-agent 6.13.0.51.gf3fde108

Add Comment

[Melissa Stone (JIRA)]

Melissa Stone commented on PUP-8338

Re: Puppet apply doesn't stream file content

Josh Cooper could you add release notes to this ticket?

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-8338

Puppet apply doesn't stream file content

Change By: Josh Cooper Release Notes Summary: This changes "puppet apply" to retrieve content for file resources with a "source" parameter in the same way that "puppet agent" does. Previously, "puppet apply" read the entire file contents into memory and cached it for the duration of the run. It also did not follow HTTP redirects or handle HTTP errors in the same way, such as HTTP service unavailable (503). Now "puppet apply" uses the same code path as "puppet agent" to stream file content from "puppet://" and "http(s)://" file sources. As a result, memory  footprint should be reduced when using "puppet apply". Release Notes: Bug Fix

Add Comment