Jira (PUP-9971) Add ETag support to the source attribute of the file resource type for http

[beliys (JIRA)]

beliys created an issue

Puppet / PUP-9971 Add ETag support to the source attribute of the file resource type for http Issue Type: New Feature Assignee: Unassigned Created: 2019/08/21 3:57 AM Priority: Normal Reporter: beliys Now according to the documentation: The http source uses the server Content-MD5 header as a checksum to determine if the remote file has changed. If the server response does not include that header, Puppet defaults to using the Last-Modified header. But github does not use any of these headers, and it would be nice to check the ETag header for efficient file downloads from there. An example of the current headers for a file with github: curl -I https://raw.githubusercontent.com/major/MySQLTuner-perl/master/mysqltuner.pl HTTP/1.1 200 OK Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'; sandbox Strict-Transport-Security: max-age=31536000 X-Content-Type-Options: nosniff X-Frame-Options: deny X-XSS-Protection: 1; mode=block ETag: "fe63f5cd29814a03d94e68e5f503e2b0b6d65bfe" Content-Type: text/plain; charset=utf-8 Cache-Control: max-age=300 X-Geo-Block-List: X-GitHub-Request-Id: E292:4560:96247:C7C3E:5D5D220C Content-Length: 221615 Accept-Ranges: bytes Date: Wed, 21 Aug 2019 10:54:49 GMT Via: 1.1 varnish Connection: keep-alive X-Served-By: cache-ams21025-AMS X-Cache: HIT X-Cache-Hits: 1 X-Timer: S1566384890.584825,VS0,VE1 Vary: Authorization,Accept-Encoding Access-Control-Allow-Origin: * X-Fastly-Request-ID: 837bdb865032ad409e8685b36f6ccfa87832832e Expires: Wed, 21 Aug 2019 10:59:49 GMT Source-Age: 236 Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[beliys (JIRA)]

beliys updated an issue

Puppet / PUP-9971 Add ETag support to the source attribute of the file resource type for http

Change By: beliys Now according to the documentation (https : //puppet.com/docs/puppet/6.7/types/file.html#file-attribute-source): {code:java} The http source uses the server Content-MD5 header as a checksum to determine if the remote file has changed. If the server response does not include that header, Puppet defaults to using the Last-Modified header.{code}

But github does not use any of these headers, and it would be nice to check the ETag header for efficient file downloads from there. An example of the current headers for a file with github:

{code:java}

Source-Age: 236{code}

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-9971 Add ETag support to the source attribute of the file resource type for http

Change By: Josh Cooper Team: Coremunity

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-9971

Re: Add ETag support to the source attribute of the file resource type for http If he desired checksum type and value are specified in the manifest, which is fairly common, then the need for ETag goes away. So I’m going to close this as won’t do.

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[beliys (Jira)]

beliys commented on PUP-9971

Re: Add ETag support to the source attribute of the file resource type for http

Can you show an example of how I can do without ETag in the example above?

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-9971

Re: Add ETag support to the source attribute of the file resource type for http

file { '/tmp/file.txt': ensure => file, source => 'http://httpstat.us/200', checksum => 'sha256', checksum_value => 'f9bafc82ba5f8fb02b25020d66f396860604f496ca919480147fa525cb505d88', }

Add Comment

[beliys (Jira)]

beliys commented on PUP-9971

Re: Add ETag support to the source attribute of the file resource type for http

Josh Cooper Thanks, but what should I do if I always want to receive the latest version of the file from the link but download only when it has changed. This method will not work for files whose checksums will change over time.

Add Comment