Jira (PUP-10792) How to set token for https url when using puppet file resource

[liuwenhe (Jira)]

liuwenhe created an issue

Puppet / PUP-10792 How to set token for https url when using puppet file resource Issue Type: Bug Affects Versions: PUP 5.5.17 Assignee: Unassigned Created: 2020/11/23 11:38 PM Priority: Normal Reporter: liuwenhe Puppet Version: 5.5.17 Puppet Server Version: 5.3.14 OS Name/Version:  CentOS Linux release 7.6.1810 (Core) Describe your issue in as much detail as possible… Describe steps to reproduce… Desired Behavior: I use Azure blob as the download source,and used SAS token. So the downloaded link is similar to 'https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3'.   Actual Behavior:   I defined a resource :   file { '1.txt': {{ path => /opt/1.txt,}} {{ source =>  'https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3', ensure => present,}} {{ }}} Then I ran puppet on the client, and I got an error. 'Error: /Stage[main]/Base::Init/File[1.txt]: Could not evaluate: Could not retrieve information from environment production source(s)'. This url cannot be downloaded directly using wget on centos7.must use wget 'url' Can anyone give me some help, thanks Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[liuwenhe (Jira)]

liuwenhe updated an issue

Puppet / PUP-10792 How to set token for https url when using puppet file resource

Change By: liuwenhe *Puppet Version: 5.5.17* *Puppet Server Version: 5.3.14* *OS Name/Version:  CentOS Linux release 7.6.1810 (Core)*

Describe your issue in as much detail as possible… Describe steps to reproduce…

*Desired Behavior:* *I use Azure blob as the download source,and used SAS token. So the downloaded link is similar to '[https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3'.|https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3%27.]*   *Actual Behavior:*

I defined a resource :   {{file { '1.txt':}}

{{       path => /opt/1.txt, }} {{     source =>  'https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3',     ensure => present, }} {{ } }} Then I ran puppet on the client, and I got an error. 'Error: /Stage[main]/Base::Init/File[1.txt]: Could not evaluate: Could not retrieve information from environment production source(s)'. This url cannot be downloaded directly using wget on centos7.must use wget 'url' Can anyone give me some help, thanks

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-10792 How to set token for https url when using puppet file resource

Change By: Josh Cooper

*Puppet Version: 5.5.17* *Puppet Server Version: 5.3.14* *OS Name/Version:  CentOS Linux release 7.6.1810 (Core)* Describe your issue in as much detail as possible… Describe steps to reproduce… *Desired Behavior:* *I use Azure blob as the download source,and used SAS token. So the downloaded link is similar to ' [https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3'.|https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3%27.] *   *Actual Behavior:*   I defined a resource :

{ { code:puppet}

file { '1.txt': }}  path => /opt/1.txt, source =>  'https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3', ensure => present, }

{code }

Then I ran puppet on the client, and I got an error.

' {noformat}

Error: /Stage[main]/Base::Init/File[1.txt]: Could not evaluate: Could not retrieve information from environment production source(s) '.

{noformat}

This url cannot be downloaded directly using wget on centos7.must use wget 'url' Can anyone give me some help, thanks

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-10792 How to set token for https url when using puppet file resource Change By: Josh Cooper

*Puppet Version: 5.5.17* *Puppet Server Version: 5.3.14* *OS Name/Version:  CentOS Linux release 7.6.1810 (Core)* Describe your issue in as much detail as possible… Describe steps to reproduce… *Desired Behavior:* *I use Azure blob as the download source,and used SAS token. So the downloaded link is similar to [https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3 '.|https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3%27.

]

*Actual Behavior:* I defined a resource : {code:puppet} file { '1.txt':  path => /opt/1.txt, source =>  'https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3', ensure => present, } {code} Then I ran puppet on the client, and I got an error. {noformat} Error: /Stage[main]/Base::Init/File[1.txt]: Could not evaluate: Could not retrieve information from environment production source(s)

{noformat} This url cannot be downloaded directly using wget on centos7.must use wget 'url' Can anyone give me some help, thanks

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10792

Re: How to set token for https url when using puppet file resource This is likely a duplicate of one or more bugs that were fixed in the 6.x series (see PUP-6380, PUP-8300, PUP-9109, PUP-10603). Puppet should now follow redirects with query parameters correctly. Can you try with the most recent 6.19.1 agent and run the agent as puppet agent -t --http_debug and include the output as a comment.

Add Comment

[liuwenhe (Jira)]

liuwenhe updated an issue

Puppet / PUP-10792

How to set token for https url when using puppet file resource

Change By: liuwenhe Attachment: image-2020-12-04-10-06-11-639.png

Add Comment

[liuwenhe (Jira)]

liuwenhe updated an issue

Puppet / PUP-10792 How to set token for https url when using puppet file resource

Change By: liuwenhe Attachment: error.txt

Add Comment

[liuwenhe (Jira)]

liuwenhe commented on PUP-10792

Re: How to set token for https url when using puppet file resource hi Josh  Thanks for your reply, I tried to install puppet7-agent on the client,Then I run puppet agent -t --http_debug But the error still occurs, the following is a screenshot, the attachment is the error content (My puppet server is still 5.3.14)   error.txt

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10792

Re: How to set token for https url when using puppet file resource

Does curl -vv 'https://test.blob.core.chinacloudapi.cn/test/key_password.txt?sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tL3q8AtFATbw1HdF/zXhp0%3' work? Can you include the output?

Add Comment

[liuwenhe (Jira)]

liuwenhe commented on PUP-10792

Re: How to set token for https url when using puppet file resource

hi Josh , the following is a  "curl -vv 'https://cashmstraccount.blob.core.chinacloudapi.cn/test/key_password.txt&sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tLApJtZMB3q8AtFATbw1mWAdIr61HdF/zXhp0%3D' "  output [root@CNC909021 tmp]# curl -vv 'https://cashmstraccount.blob.core.chinacloudapi.cn/test/key_password.txt&sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tLApJtZMB3q8AtFATbw1mWAdIr61HdF/zXhp0%3D' * About to connect() to cashmstraccount.blob.core.chinacloudapi.cn port 443 (#0) * Trying 40.73.81.132... * Connected to cashmstraccount.blob.core.chinacloudapi.cn (40.73.81.132) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 * Server certificate: * subject: CN=*.blob.core.chinacloudapi.cn,O=Shanghai Blue Cloud Technology Co. Ltd,ST=Shanghai,C=CN * start date: Sep 14 00:00:00 2020 GMT * expire date: Sep 14 12:00:00 2021 GMT * common name: *.blob.core.chinacloudapi.cn * issuer: CN=DigiCert Basic RSA CN CA G2,O=DigiCert Inc,C=US > GET /test/key_password.txt&sv=2020-02-10&si=1&sr=c&sig=t9ZbB0tLApJtZMB3q8AtFATbw1mWAdIr61HdF/zXhp0%3D HTTP/1.1 > User-Agent: curl/7.29.0 > Host: cashmstraccount.blob.core.chinacloudapi.cn > Accept: */* > < HTTP/1.1 404 The specified resource does not exist. < Content-Length: 223 < Content-Type: application/xml < Server: Blob Service Version 1.0 Microsoft-HTTPAPI/2.0 < x-ms-request-id: 2954b824-201e-001b-1b43-cc7190000000 < Date: Mon, 07 Dec 2020 02:46:20 GMT < <?xml version="1.0" encoding="utf-8"?><Error><Code>ResourceNotFound</Code><Message>The specified resource does not exist. RequestId:2954b824-201e-001b-1b43-cc7190000000 * Connection #0 to host cashmstraccount.blob.core.chinacloudapi.cn left intact Time:2020-12-07T02:46:20.9625706Z</Message></Error>

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10792

Re: How to set token for https url when using puppet file resource

We always recommend upgrading puppetserver/puppetdb prior to agents, see https://puppet.com/docs/puppet/latest/upgrade_minor.html About slowness, you might want to use wireshark and compare network traces of puppet vs curl. It's possible ruby's Net::HTTP does something inefficiently when handling chunked encoding, like read from the socket one byte at a time.

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10792

Re: How to set token for https url when using puppet file resource

liuwenhe any update on this?

Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-10792

Re: How to set token for https url when using puppet file resource

We haven't received a reply so I'm going to close this. Please reopen with the output of puppet agent --t --http_debug if the problem is reproducible.

Add Comment