Jira (PUP-10603) File Resource HTTP GET - Misuse of URL encoding

36 views
Skip to first unread message

Christopher Hughes (Jira)

unread,
Jul 28, 2020, 4:54:03 PM7/28/20
to puppe...@googlegroups.com
Christopher Hughes updated an issue
 
Puppet / Bug PUP-10603
File Resource HTTP GET - Misuse of URL encoding
Change By: Christopher Hughes
Summary: {brief summary File Resource HTTP GET - Misuse of issue} URL encoding
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Josh Cooper (Jira)

unread,
Jul 28, 2020, 7:53:04 PM7/28/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10603
 
Re: File Resource HTTP GET - Misuse of URL encoding

What does the dummy file resource look like in the manifest?

Can you include the http_debug output when using 6.17.0 (specifically due to https://tickets.puppetlabs.com/browse/PUP-10580)?

Christopher Hughes (Jira)

unread,
Jul 29, 2020, 12:51:04 AM7/29/20
to puppe...@googlegroups.com

 

if you can instruct me on the specific debug output I will go ahead and add that to the ticket.

 

 

Josh Cooper (Jira)

unread,
Jul 29, 2020, 12:59:03 AM7/29/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10603

Ah run ‘puppet agent -t —http_debug’

Christopher Hughes (Jira)

unread,
Jul 29, 2020, 3:00:04 PM7/29/20
to puppe...@googlegroups.com
  path => 'C:\test.txt',}

results in:

Notice: Compiled catalog for chugsdell in environment production in 0.02 seconds
opening connection to anystorage.blob.core.windows.net:443...
opened
starting SSL for anystorage.blob.core.windows.net:443...
SSL established
<- "HEAD /puppet/test/test.txt?sv=2019-02-02&st=2020-07-29T18%3A37%3A18Z&se=2020-07-29T19%3A00%3A00Z&sr=b&sp=r&sig=kGZ5nUC7mEJwVmTnJRqHG7xnU%2BbgvO83y%2BYZEBOl%2Fvk%3D HTTP/1.1\r\nX-Puppet-Version: 6.17.0\r\nUser-Agent: Puppet/6.17.0 Ruby/2.5.8-p224 (x64-mingw32)\r\nAccept: */*\r\nHost: anystorage.blob.core.windows.net\r\n\r\n"
-> "HTTP/1.1 200 OK\r\n"
-> "Content-Length: 4\r\n"
-> "Content-Type: text/plain; charset=utf-8\r\n"
-> "Content-MD5: kS7IA7LOSeSlQQaNSVq1cA==\r\n"
-> "Last-Modified: Tue, 28 Jul 2020 20:32:32 GMT\r\n"
-> "Accept-Ranges: bytes\r\n"
-> "ETag: \"0x8D833355AE20F01\"\r\n"
-> "Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0\r\n"
-> "x-ms-request-id: 68fea29e-501e-0093-08d9-65bab7000000\r\n"
-> "x-ms-version: 2019-02-02\r\n"
-> "x-ms-creation-time: Tue, 28 Jul 2020 20:32:32 GMT\r\n"
-> "x-ms-lease-status: unlocked\r\n"
-> "x-ms-lease-state: available\r\n"
-> "x-ms-blob-type: BlockBlob\r\n"
-> "Content-Disposition: \r\n"
-> "x-ms-server-encrypted: true\r\n"
-> "Date: Wed, 29 Jul 2020 18:53:46 GMT\r\n"
-> "\r\n"
Conn keep-alive
<- "GET /puppet/test/test.txt?sv=2019-02-02&st=2020-07-29T18%253A37%253A18Z&se=2020-07-29T19%253A00%253A00Z&sr=b&sp=r&sig=kGZ5nUC7mEJwVmTnJRqHG7xnU%252BbgvO83y%252BYZEBOl%252Fvk%253D HTTP/1.1\r\nX-Puppet-Version: 6.17.0\r\nUser-Agent: Puppet/6.17.0 Ruby/2.5.8-p224 (x64-mingw32)\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nHost: anystorage.blob.core.windows.net\r\n\r\n"
-> "HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\r\n"
-> "Content-Length: 408\r\n"
-> "Content-Type: application/xml\r\n"
-> "Server: Microsoft-HTTPAPI/2.0\r\n"
-> "x-ms-request-id: 68fea2ce-501e-0093-30d9-65bab7000000\r\n"
-> "x-ms-error-code: AuthenticationFailed\r\n"
-> "Date: Wed, 29 Jul 2020 18:53:46 GMT\r\n"
-> "\r\n"
reading 408 bytes...
-> "\xEF\xBB\xBF<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:68fea2ce-501e-0093-30d9-65bab7000000\nTime:2020-07-29T18:53:47.2926554Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error>"
read 408 bytes
Conn close because of error Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Error: Could not set 'file' on ensure: Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:68fea2ce-501e-0093-30d9-65bab7000000
Time:2020-07-29T18:53:47.2926554Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error> (file: C:/Users/chughes/Documents/Source/puppet/test.pp, line: 4)
Error: Could not set 'file' on ensure: Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:68fea2ce-501e-0093-30d9-65bab7000000
Time:2020-07-29T18:53:47.2926554Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error> (file: C:/Users/chughes/Documents/Source/puppet/test.pp, line: 4)
Wrapped exception:
Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:68fea2ce-501e-0093-30d9-65bab7000000
Time:2020-07-29T18:53:47.2926554Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error>
Error: /Stage[main]/Main/File[test.txt]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:68fea2ce-501e-0093-30d9-65bab7000000
Time:2020-07-29T18:53:47.2926554Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error> (file: C:/Users/chughes/Documents/Source/puppet/test.pp, line: 4)

url unencoded

  path => 'C:\test.txt',}

results in:

Notice: Compiled catalog for chugsdell in environment production in 0.02 seconds
opening connection to anystorrage.blob.core.windows.net:443...
opened
starting SSL for anystorage.blob.core.windows.net:443...
SSL established
<- "HEAD /puppet/test/test.txt?sv=2019-02-02&st=2020-07-29T18:37:18Z&se=2020-07-29T19:00:00Z&sr=b&sp=r&sig=kGZ5nUC7mEJwVmTnJRqHG7xnU+bgvO83y+YZEBOl/vk= HTTP/1.1\r\nX-Puppet-Version: 6.17.0\r\nUser-Agent: Puppet/6.17.0 Ruby/2.5.8-p224 (x64-mingw32)\r\nAccept: */*\r\nHost: anystorage.blob.core.windows.net\r\n\r\n"
-> "HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\r\n"
-> "Transfer-Encoding: chunked\r\n"
-> "Server: Microsoft-HTTPAPI/2.0\r\n"
-> "x-ms-request-id: 7dc31fcb-801e-0134-74da-65c501000000\r\n"
-> "x-ms-error-code: AuthenticationFailed\r\n"
-> "Date: Wed, 29 Jul 2020 18:57:33 GMT\r\n"
-> "\r\n"
Conn keep-alive
<- "GET /puppet/test/test.txt?sv=2019-02-02&st=2020-07-29T18:37:18Z&se=2020-07-29T19:00:00Z&sr=b&sp=r&sig=kGZ5nUC7mEJwVmTnJRqHG7xnU+bgvO83y+YZEBOl/vk= HTTP/1.1\r\nX-Puppet-Version: 6.17.0\r\nUser-Agent: Puppet/6.17.0 Ruby/2.5.8-p224 (x64-mingw32)\r\nRange: bytes=0-0\r\nAccept: */*\r\nHost: anystorage.blob.core.windows.net\r\n\r\n"
-> "HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\r\n"
-> "Content-Length: 408\r\n"
-> "Content-Type: application/xml\r\n"
-> "Server: Microsoft-HTTPAPI/2.0\r\n"
-> "x-ms-request-id: 7dc31fd7-801e-0134-7eda-65c501000000\r\n"
-> "x-ms-error-code: AuthenticationFailed\r\n"
-> "Date: Wed, 29 Jul 2020 18:57:33 GMT\r\n"
-> "\r\n"
reading 408 bytes...
-> "\xEF\xBB\xBF<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:7dc31fd7-801e-0134-7eda-65c501000000\nTime:2020-07-29T18:57:34.2913525Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error>"
read 408 bytes
Conn keep-alive
Error: /Stage[main]/Main/File[test.txt]: Could not evaluate: Could not retrieve information from environment production source(s) https://anystorage.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-29T18:37:18Z&se=2020-07-29T19:00:00Z&sr=b&sp=r&sig=kGZ5nUC7mEJwVmTnJRqHG7xnU+bgvO83y+YZEBOl/vk=
Notice: Applied catalog in 0.63 seconds

Josh Cooper (Jira)

unread,
Aug 5, 2020, 11:28:02 AM8/5/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Aug 5, 2020, 11:28:03 AM8/5/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Platform Core KANBAN

Josh Cooper (Jira)

unread,
Aug 5, 2020, 11:28:03 AM8/5/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10603
 
Re: File Resource HTTP GET - Misuse of URL encoding

Yes it appears metadata and content are not handling URL encoding the same way. I'll take a look at this.

Josh Cooper (Jira)

unread,
Aug 5, 2020, 11:28:03 AM8/5/20
to puppe...@googlegroups.com
Josh Cooper assigned an issue to Josh Cooper
 
Change By: Josh Cooper
Assignee: Josh Cooper

Josh Cooper (Jira)

unread,
Aug 7, 2020, 7:53:03 PM8/7/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.18.0

Josh Cooper (Jira)

unread,
Aug 7, 2020, 7:54:03 PM8/7/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Aug 18, 2020, 12:06:03 PM8/18/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Aug 18, 2020, 12:09:05 PM8/18/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Release Notes: Bug Fix
Release Notes Summary: Puppet 6.16.0 introduced a regression when using http file sources and the URL request path or query contained special characters. This restores the behavior that existed previously. In order to specify a path or query that contains a space (for example), then it must be URL encoded in the manifest, e.g. source => 'http://example.com/path%20to%file?q=a%20b'

Claire Cadman (Jira)

unread,
Aug 20, 2020, 4:59:03 AM8/20/20
to puppe...@googlegroups.com
Claire Cadman updated an issue
Change By: Claire Cadman
Labels: docs-reviewed

Josh Cooper (Jira)

unread,
Aug 20, 2020, 2:07:04 PM8/20/20
to puppe...@googlegroups.com

Christopher Hughes (Jira)

unread,
Aug 24, 2020, 10:24:04 AM8/24/20
to puppe...@googlegroups.com

Hey Josh Cooper  sorry for the delay,

 

I unfortunately don't have an environment to run beta server versions, and have never tried. Can you point me in a direction that would provide some insight to this?

Josh Cooper (Jira)

unread,
Aug 24, 2020, 12:47:03 PM8/24/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10603

You can install the nightly puppet6 version (for example for yum http://nightlies.puppet.com/yum/puppet6-nightly/), or wait for the 6.18 release coming out this week.

Reply all
Reply to author
Forward
0 new messages